-
2008-12-09
Changing the host name of LINUX, Xinetd, network commands, PAM, network monitor - [UNIX/LINUX]
Changing the host name: This is a three step process:
- Issue the command: hostname new-host-name
- Change network configuration file: /etc/sysconfig/network
Edit entry: HOSTNAME=new-host-name - Restart systems which relied on the hostname (or reboot):
- Restart network services: service network restart
(or: /etc/init.d/network restart) - Restart desktop:
- Bring down system to console mode: init 3
- Bring up X-Windows: init 5
- Restart network services: service network restart
The hostname may be changed at runtime using the command: sysctl -w kernel.hostname="superserver"
Change the host name using GUI tool: /usr/sbin/system-config-network
(Red Hat / Fedora / CentOS)Hostname entries are made in two places:
Select the "DNS" tab. Select the "Devices" tab + "Edit" + the "General" tab. Network IP aliasing: Assign more than one IP address to one ethernet card:
ifconfig eth0 XXX.XXX.XXX.XXX netmask 255.255.255.0 broadcast XXX.XXX.XXX.255
ifconfig eth0:0 192.168.10.12 netmask 255.255.255.0 broadcast 192.168.10.255
ifconfig eth0:1 192.168.10.14 netmask 255.255.255.0 broadcast 192.168.10.255
route add -host XXX.XXX.XXX.XXX dev eth0
route add -host 192.168.10.12 dev eth0
route add -host 192.168.10.14 dev eth0In this example 0 and 1 are aliases in addition to the regular eth0. The result of the ifconfig command:
eth0 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:XXX.XXX.XXX.XXX Bcast:XXX.XXX.XXX.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14218 errors:0 dropped:0 overruns:0 frame:0
TX packets:1362 errors:0 dropped:0 overruns:0 carrier:0
collisions:1 txqueuelen:100
Interrupt:5 Base address:0xe400
eth0:0 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:192.168.10.12 Bcast:192.168.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:5 Base address:0xe400
eth0:1 Link encap:Ethernet HWaddr 00:10:4C:25:7A:3F
inet addr:192.168.10.14 Bcast:192.168.10.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:5 Base address:0xe400Config file: /etc/sysconfig/network-scripts/ifcfg-eth0:0
DEVICE=eth0:0
ONBOOT=yes
BOOTPROTO=static
BROADCAST=192.168.10.255
IPADDR=192.168.10.12
NETMASK=255.255.255.0
NETWORK=192.168.10.0
ONBOOT=yes
Aliases can also be shut down independently. i.e.: ifdown eth0:0
The option during kernel compile is: CONFIG_IP_ALIAS=y (Enabled by default in Redhat)
Note: The Apache web server can be configured so that different IP addresses can be assigned to specific domains being hosted. See Apache configuration and "configuring an IP based virtual host" in the YoLinux Web site configuration tutorial.
Network Classes: The concept of network classes is a little obsolete as subnets are now used to define smaller networks. These subnets may be part of a class A, B, C, etc network. For historical reference the network classes are defined as follows:
- Class A: Defined by the first 8 bits with a range of 0 - 127.
First number (8 bits) is defined by Internic i.e. 77.XXX.XXX.XXX
One class A network can define 16,777,214 hosts.
Range: 0.0.0.0 - 127.255.255.255 - Class B: Defined by the first 8 bits with a range from 128 - 191
First two numbers (16 bits) are defined by Internic i.e. 182.56.XXX.XXX
One class B network can define 65,534 hosts.
Range: 128.0.0.0 - 191.255.255.255 - Class C: Defined by the first 8 bits with a range from 192 - 223
First three numbers (24 bits) are defined by Internic i.e. 220.56.222.XXX
One class B network can define 254 hosts.
Range: 192.0.0.0 - 223.255.255.255 - Class D: Defined by the first 8 bits with a range from 224 - 239
This is reserved for multicast networks (RFC988)
Range: 224.0.0.0 - 239.255.255.255 - Class E: Defined by the first 8 bits with a range from 240 - 255
This is reserved for experimental use.
Range: 240.0.0.0 - 247.255.255.255
Forwarding allows the network packets on one network interface (i.e. eth0) to be forwarded to another network interface (i.e. eth1). This will allow the Linux computer to conect ("ethernet bridge") or route network traffic.Enable Forwarding: The bridge configuration will merge two (or several) networks into one single network topology. IpTables firewall rules can be used to filter traffic.
A router configuration can support multicast and basic IP routing using the "route" command. IP masquerading (NAT) can be used to connect private local area networks (LAN) to the internet or load balance servers.
- Turn on IP forwarding to allow Linux computer to act as a gateway or router.
echo 1 > /proc/sys/net/ipv4/ip_forward
Default is 0. One can add firewall rules by using ipchains.Another method is to alter the Linux kernel config file: /etc/sysctl.conf Set the following value:
net.ipv4.ip_forward = 1
See file /etc/sysconfig/network for storing this configuration.
FORWARD_IPV4=true
All methods will result in a proc file value of "1". Test: cat /proc/sys/net/ipv4/ip_forward
The TCP Man page - Linux Programmer's Manual and /usr/src/linux/Documentation/proc.txt (Kernel 2.2 RH 7.0-) cover /proc/sys/net/ipv4/* file descriptions.
Alos see: (YoLinux tutorials)
- Configure Linux as an internet gateway router: Using Linux and iptables/ipchains to set up an internet gateway for home or office (iptables)
- Load balancing servers using LVS (Linux Virtual Server) (ipvsadm)
Usefull Linux networking commands: - /etc/rc.d/init.d/network start - command to start, restart or stop the network
- netstat - Display connections, routing tables, stats etc
- List externally connected processes: netstat -punta
- List all connected processes: netstat -nap
- Show network statistics: netstat -s
- Kernel interface table info: netstat -a -i eth0
- ping - send ICMP ECHO_REQUEST packets to network hosts. Use Cntl-C to stop ping.
- traceroute - print the route packets take to network host
- traceroute IP-address-of-server
- traceroute domain-name-of-server
- mtr - a network diagnostic tool introduced in Fedora - Like traceroute except it gives more network quality and network diagnostic info. Leave running to get real time stats. Reports best and worst round trip times in milliseconds.
- mtr IP-address-of-server
- mtr domain-name-of-server
- whois - Lookup a domain name in the internic whois database.
- finger - Display information on a system user. i.e. finger user@host Uses $HOME/.plan and $HOME/.project user files. Often used by game developers. See http://finger.planetquake.com/
- iptables - IP firewall administration (Linux kernel 2.6/2.4) See YoLinux firewall/gateway configuration.
- ipchains - IP firewall administration (Linux kernel 2.2) See YoLinux firewall/gateway configuration.
- socklist - Display list of open sockets, type, port, process id and the name of the process. Kill with fuser or kill.
- host - Give a host name and the command will return IP address. Unlike nslookup, the host command will use both /etc/hosts as well as DNS.
Example: host domain-name-of-server - nslookup - Give a host name and the command will return IP address. Also see Testing your DNS (YoLinux Tutorial) Note that nslookup does not use the /etc/hosts file.
inetd/xinetd: Network Socket Listener Daemons: The network listening daemons listen and respond to all network socket connections made on the TCP/IP ports assigned to it. The ports are defined by the file /etc/services. When a connection is made, the listener will attempt to invoke the assigned program and pipe the data to it. This simplified matters by allowing the assigned program to read from stdin instead of making its own sockets connection. The listener hadles the network socket connection. Two network listening and management daemons have been used in Red Hat Linux distributions:
- inetd: Red Hat 6.x and older
- xinetd: Red Hat 7.0-9.0, Fedora Core
inetd: - Configuration file: /etc/inetd.conf
Entries in this file consist of a single line made up of the following fields:service socket-type protocol wait user server cmdline
- service: The name assigned to the service. Matches the name given in the file /etc/services
- socket-type:
- stream: connection protocols (TCP)
- dgram: datagram protocols (UDP)
- raw
- rdm
- seqpacket
- protocol: Transport protocol name which matches a name in the file /etc/protocols. i.e. udp, icmp, tcp, rpc/udp, rpc/tcp, ip, ipv6
- wait: Applies only to datagram protocols (UDP).
- wait[.max]: One server for the specified port at any time (RPC)
- nowait[.max]: Continue to listen and launch new services if a new connection is made. (multi-threaded)
- user[.group]: login id of the user the process is executed under. Often nobody, root or a special restricted id for that service.
- server: Full path name of the server program to be executed.
- cmdline: Command line to be passed to the server. This includes argument 0 (argv[0]), that is the command name. This field is empty for internal services. Example of internal TCP services: echo, discard, chargen (character generator), daytime (human readable time), and time (machine readable time). (see RFC)
Sample File: /etc/inetd.conf
#echo stream tcp nowait root internal
#echo dgram udp wait root internal
ftp stream tcp nowait root /usr/sbin/tcpd in.ftpd -l -a
#pop-3 stream tcp nowait root /usr/sbin/tcpd ipop3d
#swat stream tcp nowait.400 root /usr/sbin/swat swat
The inet daemon must be restarted to pick up the changes made to the file:
/etc/rc.d/init.d/inetd restartFor more information see the man pages "inetd" and "inetd.conf".
xinetd: Extended Internet Services Daemon: - Xinetd has access control machanisms, logging capabilities, the ability to make services available based on time, and can place limits on the number of servers that can be started, redirect services to different ports and network interfaces (NIC) or even to a different server, chroot a service etc... and thus a worthy upgrade from inetd.
Use the command chkconfig --list to view all system services and their state. It will also list all network services controlled by xinetd and their respective state under the title "xinetd based services". (Works for xinetd (RH7.0+) but not inetd)
The xinetd network daemon uses PAM also called network wrappers which invoke the /etc/hosts.allow and /etc/hosts.deny files.
Configuration file: /etc/xinetd.conf which in turn uses configuration files found in the directory /etc/xinetd.d/.
To turn a network service on or off:
- Edit the file /etc/xinetd.d/service-name
Set the disable value:- disable = yes
or
disable = no
- SIGUSR1 (kill -SIGUSR1 process-id) - Soft reconfiguration does not terminate existing connections. (Important if you are connected remotely)
- SIGUSR2 - Hard reconfiguration stops and restarts the xinetd process.
OR - disable = yes
- Use the chkconfig command: chkconfig service-name on
(or off)
This command will also restart the xinetd process to pick up the new configuration.
The file contains entries of the form:
service service-name { attribute assignment-operator value value ... ... {Where:- attribute:
- disable:
- yes
- no
- type:
- RPC
- INTERNAL:
- UNLISTED: Not found in /etc/rpc or /etc/services
- id: By default the service id is the same as the service name.
- socket_type:
- stream: TCP
- dgram: UDP
- raw: Direct IP access
- seqpacket: service that requires reliable sequential datagram transmission
- flags: Combination of: REUSE, INTERCEPT, NORETRY, IDONLY, NAMEINARGS, NODELAY, DISABLE, KEEPALIVE, NOLIBWRAP.
See the xinetd man page for details. - protocol: Transport protocol name which matches a name in the file /etc/protocols.
- wait:
- no: multi-threaded
- yes: single-threaded - One server for the specified port at any time (RPC)
- user: See file : /etc/passwd
- group: See file : /etc/group
- server: Program to execute and recieve data stream from socket. (Fully qualified name - full pathe name of program)
- server_args: Unlike inetd, arg[0] or the name of the service is not passed.
- only_from: IP address, factorized address, netmask range, hostname or network name from file /etc/networks.
- no_access: Deny from ... (inverse of only_from)
- access_times
- port: See file /etc/services
The best source of information is the man page and its many examples. - disable:
- assignment-operator:
- =
- +=: add a value to the set of values
- -=: delete a value from the set of values
Then restart the daemon: /etc/rc.d/init.d/xinetd restart
Example from man page: Limit telnet sessions to 8 Mbytes of memory and a total 20 CPU seconds for child processes.
service telnet
{
socket_type = stream
wait = no
nice = 10
user = root
server = /usr/etc/in.telnetd
rlimit_as = 8M
rlimit_cpu = 20
}
[Pitfall] Red Hat 7.1 with updates as of 07/06/2001 required that I restart the xinetd services before FTP would work properly even though xinetd had started without failure during the boot sequence. I have no explanation as to why this occurs or how to fix it other than to restart xinetd: /etc/rc.d/init.d/xinetd restart.
Man Pages:
For more info see:
- LinuxFocus.org: xinetd - Frederic Raynal
- RedHat.com: Controlling Access to Services
- http://www.xinetd.org/
- See RFC's: 862, 863, 864, 867, 868, 1413.
- man page xinetd, xinetd.conf, xinetd.log
- Edit the file /etc/xinetd.d/service-name
PAM: Network Wrappers: Pluggable Authentication Modules for Linux (TCP Wrappers)
This system allows or denies network access. One can reject or allow specific IP addresses or subnets to access your system.
File: /etc/hosts.allow
in.ftpd:208.188.34.105
This specifically allows the given IP address to ftp to your system. One can also specify an entire domain. i.e. .name-of-domain.com
Note the beginning ".".File: /etc/hosts.deny
ALL:ALL
This generally denies any access.
File: /etc/inetd.conf
ftp stream tcp nowait root /usr/sbin/tcpd in.ftpd -l -a
The inet daemon accepts the incoming network stream and assigns it to the PAM TCP wrapper, /usr/sbin/tcpd, which accepts or denies the network connection as defined by /etc/hosts.allow and /etc/hosts.deny and then passes it along to ftp. This is logged to /var/log/secure
Advanced PAM: More specific access can be assigned and controlled by controlling the level of authentication required for access.
Files reflect the inet service name. Rules and modules are stacked to achieve the level of security desired.
See the files in /etc/pam.d/... (some systems use /etc/pam.conf)
The format: service type control module-path module-arguments
- auth - (type) Password is required for the user
- nullok - Null or non-existatant password is acceptable
- shadow - encrypted passwords kept in /etc/shadow
- account - (type) Verifies password. Can track and force password changes.
- password - (type) Controls password update
- retry=3 - Sets the number of login attempts
- minlen=8 - Set minimum length of password
- session - (type) Controls monitoring
Modules:
- /lib/security/pam_pwdb.so - password database module
- /lib/security/pam_shells.so -
- /lib/security/pam_cracklib.so - checks is password is crackable
- /lib/security/pam_listfile.so
After re-configuration, restart the inet daemon: killall -HUP inetd
For more info see:
Network Monitoring Tools: - tcpdump - dump traffic on a network. See discussion below.
Command line option Description -c Exit after receiving count packets. -C Specify size of output dump files. -i Specify interface if multiple exist. Lowest used by default. i.e. eth0 -w file-name Write the raw packets to file rather than parsing and printing them out.
They can later be printed with the -r option.-n Improve speed by not performing DNS lookups. Report IP addresses. -t Don't print a timestamp on each dump line.
Filter expressions:primitive Description host host-name If host has multiple IP's, all will be checked. net network-number Network number. net network-number mask mask Network number and netmask specified. port port-number Port number specified. tcp Sniff TCP packets. udp Sniff UDP packets. icmp Sniff icmp packets. Examples:
- tcpdump tcp port 80 and host server-1
- tcpdump ip host server-1 and not server-2
- iptraf - Interactive Colorful IP LAN Monitor
- nmap - Network exploration tool and security scanner
- List pingable nodes on network: nmap -sP 192.168.0.0/24
Scans network for IP addresses 192.168.0.0 to 192.168.0.255 using ping.
- List pingable nodes on network: nmap -sP 192.168.0.0/24
- Ethereal - Network protocol analyzer. Examine data from a live network.
- RPM's required:
- ethereal-0.8.15-2.i386.rpm - Red Hat 7.1 Powertools CD RPM
- ucd-snmp-4.2-12.i386.rpm - Red Hat 7.1 binary CD 1
- ucd-snmp-utils-4.2-12.i386.rpm - Red Hat 7.1 binary CD 1
- Also: gtk+, glib, glibc, XFree86-libs-4.0.3-5 (base install)
- RPM's required:
- EtherApe - Graphical network monitor for Unix modeled after etherman. This is a great network discovery program with cool graphics. (Red Hat Powertools CD 7.1)
- Gkrellm - Network and system monitor. Good for monitoring your workstation. (Red Hat Powertools CD)
- IPTraf - ncurses-based IP LAN monitor. (Red Hat Powertools CD)
- Cheops - Network discovery, location, diagnosis and management. Cheops can identify all of the computers that are on your network, their IP address, their DNS name, the operating system they are running. Cheops can run a port scan on any system on your network. (Red Hat Powertools CD)
- ntop - Shows network usage in a way similar to what top does for processes. Monitors how much data is being sent and received on your network. (Red Hat Powertools CD)
- MRTG - Multi Router Traffic Grapher - Monitor network traffic load using SNMP and generate an HTML/GIF report. (See sample output)
- dnsad - IP traffic capture. Export to Cisco Netflow for network analysis reporting.
- scotty - Obtain status and configuration information about your network. Supports SNMP, ICMP, DNS, HTTP, SUN RPC, NTP, & UDP. (Red Hat Powertools CD)
- Big Brother - Monitoring ans services availablility.
- OpenNMS.org - Network Management using SNMP.
- Nagios - host, service and network monitoring
- Angel network monitor
Using tcpdump to monitor the network:
[root]# ifconfig eth0 promisc - Put nic into promiscuous mode to sniff traffic. [root]# tcpdump -n host not XXX.XXX.XXX.XXX | more - Sniff net but ignore IP which is your remote session. [root]# ifconfig eth0 -promisc - Pull nic out of promiscuous mode.
Network Intrusion and Hacker Detection Systems: SNORT: Monitor the network, performing real-time traffic analysis and packet logging on IP networks for the detection of an attack or probe.
- Linux Journal: Planning IDS for Your Enterprise - Nalneesh Gaur
- InterSect Alliance - Intrusiuon analysis. Identifies malicious or unauthorized access attempts.
ARP: Address Resolution Protocol Ethernet hosts use the Address Resolution Protocol (ARP) to convert a 32-bit internet IP addresses into a 48-bit Ethernet MAC address used by network hardware. (See: RFC 826) ARP broadcasts are sent to all hosts on the subnet by the data transmitting host to see who replies. The broadcast is ignored by all except the intended receiver which recognizes the IP address as its own. The MAC addresses are remembered (APR cache) for future network communications. Computers on the subnet typically keep a cache of ARP responses. ARP broadcasts are passed on by hubs and switches but are blocked by routers.
Reverse ARP (See: RFC 903) is a bootstrap protocol which allows a client to broadcast requesting a server to reply with its IP address.
- arp (8) man page - manipulate the system ARP cache
- Shows other systems on your network (including IP address conflicts): arp -a
- Show ARP table Linux style: arp -e
- arpwatch (8) man page - keep track of ethernet/ip address pairings
- arpsnmp (8) man page - keep track of ethernet/ip address pairings. Reads information generated by snmpwalk
- arping (8) man page - send ARP REQUEST to a neighbor host
Print ARP reply (similar to arp -a): arping 192.168.10.99 - List ARP table: cat /proc/net/arp
- ip (8) man page - show / manipulate routing, devices, policy routing and tunnels
View ARP table: ip neighbor
ARP is something that simply works. No Linux system configuration is necessary. It's all part of the ethernet and IP protocol. The aforementioned information is just part of the Linux culture of full visibility into what is going on.
Configuring Linux For Network Multicast: Regular network exchanges of data are peer to peer unicast transactions. An HTTP request to a web server (TCP/IP), email SNMP (TCP/IP), DNS (UDP), FTP (TCP/IP), ... are all peer to peer unicast transactions. If one wants to transmit a video, audio or data stream to multiple nodes with one transmission stream instead of multiple individual peer to peer connections, one for each node, one may use multicasting to reduce network load. Note that multicast and a network broadcast are different. Multicast messages are only "heard" by the nodes on the network that have "joined the multicast group" which are those that are interested in the information.
The Linux kernel is Level-2 Multicast-Compliant. It meets all requirements to send, receive and act as a router for multicast datagrams. For a process to receive multicast datagrams it has to request the kernel to join the multicast group and bind the port receiving the datagrams. When a process is no longer interested in the multicast group, a request is made to the kernel to leave the group. It is the kernel/host which joins the multicast group and not the process. Kernel configuration requires "CONFIG_IP_MULTICAST=y". In order for the Linux kernel to support multicast routing, set the following in the kernel config:
- CONFIG_IP_MULTICAST=y
- CONFIG_IP_ROUTER=y
- CONFIG_IP_MROUTE=y
- CONFIG_NET_IPIP=y
The default Red Hat / Fedora kernels are compiled to support multicast.
See the YoLinux tutorial on optimization and rebuilding the Linux kernal.
Note that on multihomed systems (more than one IP address/network card), only one device can be configured to handle multicast.
Class D networks with a range of IP addresses from 224.0.0.0 to 239.255.255.255 (See Network Classes above) have typically been reserved for multicast.
Usefull commands:
Command Description cat /proc/net/igmp List multicast group to which the host is subscribed. Use "Internet Group Management Protocol".
(See /usr/src/linux/net/core/igmp.c)cat /proc/net/dev_mcast List multicast interfaces.
(See /usr/src/linux/net/core/dev_mcast.c)ping 224.0.0.1 All hosts configured for multicast will respond with their IP addresses ping 224.0.0.2 All routers configured for multicast will respond ping 224.0.0.3 All PIM routers configured for multicast will respond ping 224.0.0.4 All DVMRP routers configured for multicast will respond ping 224.0.0.5 All OSPF routers configured for multicast will respond
Multicast transmissions are achieved through proper routing, router configuration (if communicating through subnets) and programatically with the use of the following "C" function library calls:
For more on multicast programming see: Multicast Howto.Function Call Description setsockopt() Pass information to the Kernel. getsockopt() Retrieve information broadcast using multicast. The multicast application will specify the multicast loopback interface, TTL (network time to live), network interface and the multicast group to add or drop.
- route add 224.0.0.0 netmask 240.0.0.0 dev eth0
Living in a MS/Windows World: - SMB4k: My favorite MS/Windows file share browser.
- In Nautilus use the URL "smb:" to view MS/Windows servers. [tutorial]
- LinNeighborhood: Linux workstation gui tool.
Make your life simple and use the GUI/File Manager LinNeighborhood. It uses smbmount, samba and smbclient to give you access to MS/Windows servers and printers.
See the YoLinux tutorial on integrating Linux into a Microsoft network.
Network Definitions: - IPv4: Most of the Internet servers and personal computers use Internet Protocol version 4 (IPv4). This uses 32 bits to assign a network address as defined by the four octets of an IP address up to 255.255.255.255. Which is the representation of four 8 bit numbers thus totaling 32 bits.
- IPv6: Internet Protocol version 6 (IPv6) uses a 128 bit address and thus billions and billions of potential addresses. The protocol has also been upgraded to include new quality of service features and security. Currently Linux supports IPv6 but IPv4 is used when connecting your computer to the internet.
- TCP/IP: (Transmission Control Protocol/Internet Protocol) uses a client - server model for communications. The protocol defines the data packets transmitted (packet header, data section), data integrity verification (error detection bytes), connection and acknowledgement protocol, and re-transmission.
- TCP/IP time to live (TTL): This is a counting mechanism to determine how long a packet is valid before it reaches its destination. Each time a TCP/IP packet passes through a router it will decrement its TTL count. When the count reaches zero the packet is dropped by the router. This ensures that errant routing and looping aimless packets will not flood the network.
- MAC Address: (media access control) is the network card address used for communication between other network devices on the subnet. This info is not routable. The ARP table maps TCP/IP address (global internet) to the local hardware on the local network. Use the command /sbin/ifconfig to view both the IP address and the MAC address. The MAC address uniquely identifies each node of a network and is used by the Ethernet protocol.
- Full Duplex: Allows the simultaneous sending and receiving of packets. Most modern modems support full duplex.
- Half Duplex: Allows the sending and receiving of packets in one direction at a time only.
- OSI 7 Layer Model: The ISO (International Standards Organization) has defined the OSI (Open Systems Interconnection) model for current networking protocols.
OSI Layer Description Linux Networking Use 7 Application Layer.
The top layer for communications applications like email and the web.telnet, web browser, sendmail 6 Presentation Layer.
Syntax and format of data transfer.SMTP, http 5 Session Layer. 4 Transport Layer.
Connection, acknowledgement and data packet transmission.TCP
UDP3 Network Layer. IP
ARP2 Data Link Layer.
Error control, timingEthernet 1 Physical Layer.
Electrical characteristics of signal and NICEthernet
- Network Hub: Hardware to connect network devices together. The devices will all be on the same network and/or subnet. All network traffic is shared and can be sniffed by any other node connected to the same hub.
- Network Switch: Like a hub but creates a private link between any two connected nodes when a network connection is established. This reduces the amount of network collisions and thus improves speed. Broadcast messages are still sent to all nodes.
-
2008-03-17
zt 几种Linux嵌入式开发环境的简单介绍 - [UNIX/LINUX]
做Linux嵌入式系统的对常见的几种嵌入式开发环境一定不会默生,由于主要接触网络相关产品的一些系统设计,因此,将可能用到的嵌入式开发环境简要总结一下。主要涉及下面的几个东东:emDebian - emdebian.sourceforge.net
uClinux - http://www.uclinux.org/
buildroot - buildroot.uclibc.org
scratchbox - http://www.scratchbox.org/
openEmbedded - http://oe.handhelds.org
emDebian
emDebian基于将Debian用于嵌入式系统的目的而开发。Debian是一个发展很快的项目,在我第一次用Debian时,就再也不愿意换用其它的发布版了,目前我用的Debian已经安装了有两年的时间了,但现在系统仍然是“最新”版本,良好的在线软件升级系统是Debian成功的原因之一。目前Debian已经支持11个体系的系统,包括X86、PPC、MIPS、ARM、SH等(据最近的一则消息,ARM有可能不再支持),并包含了大量的软件。这些要归功于Debian的开发团队,正因为有许多人使用和支持,因此,不是比较偏门的软件,基本上不需要从源码来安装,这也是我喜欢用 Debian的原因之一。
这样好的一个系统,当然有人愿意将其用到嵌入式系统中去。emDebian基于一个很简易的嵌入式系统开发的想法来构造嵌入式系统,即从一个成熟的系统中去除不需要的部份(如文档和不需要的工具),精简出一个小的系统,这与下面要介绍的几个工具的想法刚好相反(下面几个都是基于 from scratch 即从无到有,从头构建的方式)。emDebian提供一些工具来协助完成从现有的系统或安装包(deb文件,类似Redhat的rpm)中提取需要的东东,并协助完成完整系统的构建,当然也支持交叉构建了,比如你可以在X86 的PC上构建一个基于ARM的嵌入式系统,而整个过程不需要编译任何一行源代码。
顺理成章的,emDebian的重要优势就展现出来了,现在你用的CPU超出11个Debian支持范围了吗?没有,那么你可以简单的通过 emDebian构建目标系统;你所需要的主体软件在Debian支持的官方和非官方近2万个软件以外吗?没有,那么恭喜你,明天就可以给老板交工了。当然,对于特定的软件,可能还是需要从源码来构建,不过同样的,我们可以将其生成Deb包,然后将配置加到emDebian工具集中,同其它所有软件一样的选取和配置。
emDebian的发展似乎不是想像的那么好,现在主页上的新闻更新还是去2004年的。
buildroot
emDebian实际上并不一定适合于资源非常紧缺的超小型系统,比如只有2M Flash的小型控制系统。另外发行版的软件通常会以通用代码来编译,例如,为了尽可能在各种X86平台上都能够安装,大多数发行版通常会以i686甚至 i386代码集来编译软件,可以使文件的通用性很强,但CPU的性能却不能发恢到最好(这就是为什么有时会看到一些厂商或爱好者发布PIII、PIV、 athlon等优化系统的原因),这对于嵌入式系统来说也不会是一件好事情。另外,没有源码的控制权,一些需要定制的东西也会变得难以实现,因此,从源码开始构建仍然有必要。
嵌入式Linux开发中使用的CPU速度往往向对不会太高,因此,尽可能提高代码的性能就非常必要。通常开发人员应该对该CPU的具体型号有一定的了解,以便启用编译器中对该型号的优化,以ARM为例,我们可以通过 -march=armv5te 和 -mtune=arm9tdmi 来对代码在ARM9上的运行进行优化。有时这些优化体现出来的性能改善是比较大的,我曾对比过一些复杂算法的代码优化前后的性能(执行速度),都有一定的提升。另外在PIV上测试过以i686和pentium4对一个语音编码算法进行优化,运算速度居然提高了几倍。
这种幅度的提升可能只是一个特例,这个算法有大量的复杂浮点运算,使用i386或i686指令集和使用更先进的PIV指令集编译出来的机器代码对于同一个运算的解释可能采用完全不同的指令来完成,因此性能提升较大就不足为奇了。同样这种代码,在ARM上通过ARM4和ARM5来优化后在ARM9上运行,却没有那么大的提升。看来对CPU的一定了解也应该是嵌入式系统软件设计者应该具备的能力。
那么又如何控制可执行文件的大小呢?除了却除软件中不需要的部份外,我们还应该考虑软件所引用的库文件。GNU的Glibc是一个非常宠大而完整的库,至少对于嵌入式系统来说,其体积显得过于大了一些。uClibc的提出较好的解决了这样一个问题。uClibc尽可能的兼容Glibc,大多数应用程序可以在很小或完全不修改的情况下就可能使用uClibc替代glibc。通过uClibc来代替Glibc,可以在不改变应用程序功能的前提下,大大减少发布文件的大小,无论应用程序以静态链接来编译,还是以动态链接形式编译。
不过使用uClibc代替并不是简单的设置一两个参数就行了,通常需要使用一个不同的工具集(gcc/binutils等)来编译代源码。手工的构造这样一个环境,对于大多数普通程序员来说,不一定是一件很简单的事情,因此,uClibc的开发者创造出一个叫做buildroot的工具集。 buildroot将自动构造编译基于uClibc代码的工具集和uClibc库,并提供一个可配置的框架和一些构建一个基本系统的配置文件。用户只需要通过配置菜单选择了相应的目标软件,buildroot就可以从构建基本工具集开始,一直到最后构建出目标系统所需要的东西,如嵌入式系统常用的基于 ext2的initrd,jffs根文件系统,压缩的根目录树等,这些代码都是基于uClibc而不是系统的Glibc的。Buildroot对主机系统的要求较小,通常只需要主机系统提供足以构建工具链(toolchain)的工具,如gcc/binutils等,当工具链编译完成后,对目标系统需要的源码的编译过程与主机系统的开发工具集基本上就没有什么关系了。因此,不同的主机如果能够通过第一步,编译完成工具链,那么编译出来的目标系统的执行代码就可以几乎不存在由于系统引起的差异。这样,开发人员就可能在各自喜欢的Linux发行版上进行开发,而不必担心出现什么兼容性问题。
uClinux
uClinux与emDebian至少有两个重要的区别,第一是构建方式,前面已经提到过了,uClinux属于 from scratch 一类的。另一个不同的地方,uClinux是支持不在emDebian支持的11种CPU的,当然,这个说法不是很恰当,正确的说法是uClinux支持那些不具备MMU单元的CPU体系。uClinux的第一个目的是支持MC68328芯片,现在已经能构支持更多的CPU,如Intel i960,ARM等。不过,uClinux的主体开发团队目前已经不再支持ARM了,还好 Samsung 的 Hyok S. Choi 接过了接励棒,Linux 2.6版本的补丁可以在 uClinux/ARM2.6 找到。
uClinux之前仅是核心的一些补丁,后来发展成为一个包括核心、库、应用程序、工具和编译相关的配置文件的一个集成开发环境。与 buildroot不同的是,uClinux不编译目标系统的工具集,也就是说,相应的编译工具应该提前安装好。如,对于arm来说,需要先安装ARM交叉编译器。uClinux的编译器也需要一些补丁,其中比较重要的两个方面主要包括:
用于生成FLT文件的补丁:由于MMU的关系,uClinux不支持ELF可执行文件,这个补丁主要包括bin2flt工具包和一个ld的wrapper脚本等,用于(透明于用户)生成FLT文件;
用于支持XIP(Execute In Place)的补丁:这个补丁需要对gcc进行一些小的修改;支持XIP主要是为了解决小内存环境中运行的问题。
XIP不一定适用于每种应用环境,对于内在要求特别严格的系统来说(空间第一位,如手机要求使用片内RAM),可以通过将核心和应用程序编译为XIP 支持,然后直接在Flash上运行,内存仅用于运行时数据;而对于性能要求为主的系统(如高速网络处理器),则不能因为节省一点空间而使用XIP将程序直接在Flash上运行,这样可能会降低指令的读取速度而影响系统性能(但仍然可以使用XIP,使程序的多个实例在内存中共享代码空间,以后详细说); + FLT可执行文件支持动态链接库(目前仅m68k支持,参见 uCdot: Shared libraries under uClinux mini-HOWTO)的补丁;
uClinux的编译过程大致是,首先,通过可视配置界面(menuconfig/xconfig)选取Vendor和board(实际上是选择了一些配置文件和产品相关的文件),然后根据选择构造一个适用于target的开发环境,如生成头文件和需要的库文件(uClibc、glibc或uC- libc 以及其它一些库),然后编译核心、库、应用程序,最后将所有的输出安装到romfs目录中,根据需要生成目标平台需要的映像文件(如: romfs.img、linux.bin、rootfs.gz等)
由于一些过程细节被隐藏起来,uClinux现在的编译过程方便到只需要配置一下(make menuconfig),然后 make 就可以直接获得最终输出。不过这反倒成为一些初学者学习的一个麻烦,本文完成后,根据对本文的反馈,将进一步对uClinux进行详细介绍。
总的来说,目前的uClinux是一套主要用于无MMU核(但不限于此)的嵌入式Linux集成环境,也是一个非常好的 Linux from scratch 的示例。抛开其MMU相关的补丁,uClinux也可以作为一套用于包含MMU系统的集成开发环境,Snapgear 就是一个很好的例子。实际上,我们可以从官方的uClinux源码就可以直接编译一个支运行于X86的uClinux。
Scratchbox
Scratchbox 的故事要从buildroot讲起(这不一定是scratchbox开发者的故事,只是依据我个人的认识)。buildroot可以从头开始,先构造编译器和基本开发环境,然后根据用户配配置构造一个适用于目标平台的根文件系统。这个文件系统可以有许多用法,例如,做为initrd或通过NFS输出给目标系统使用。为了减少交叉编译软件带来的麻烦,可以配置buidroot创建一套目标系统的编译环境(Gcc、binutils、lib等),这样用户可以通过这个基本文系统在目标系统上直接本地编译软件。如果目标系统性能足够的话,buildroot的任务到此就基本结束了。对于嵌入式系统的开发者来说,在目标系统上直接编译代码却不一定都能够实现,因为多数情况下,我们的目标平台处理器性能并不会那么高,这样,我们就不得不面对一个两难的选择:
继续通过buildroot编译其它的软件,性能会高许多,但每个软件都需要进行交叉编译相关的改造;
在目标平台上编译软件,对于只有几十或几百兆的低性能核来说,编译一个核心可能会让你等上半天的时间;
有没有好的办法解决性能和交叉编译的问题呢?先分析一下通过buildroot交叉编译不能解决的问题。Buildroot只在一定程度上对目标平台进行了模拟,但仍有一些是无法实现的,例如,当目标平台不同于主机平台时,不能生成并运行目标平台的中间代码。这样,许多通过autotools (autoconf/automake)配置的软件就可能会出现问题。例如,configure 脚本有时会生成一些中间代码,并试图运行以确认开发环境中是否存在某个库文件或头文件,对于在X86上编译基于uClibc X86目标平台代码可能不会出现问题,但如果目标平台是X86以外的平台,编译就可能会中断;又如,configure脚本确认编译器是否工作,会试图编译一个包含空的主程序的代码并运行,实际一个可运行于目标平台的 a.out 确实生成了,也可以正常运行于目标平台,但是这个测试会因为 a.out 被运行在主机系统上而错误的中断。这些问题一些被 buildroot 通过补丁或复杂的 configure 参数解决了,某些中间执行文件,则通过HOSTCC(主机上的CC)来生成并运行以生成最终文件。目前buildroot包含的软件或多或少都会有一些这样的补丁,而且开发者一旦深入到对软件的定制,就会不停的被这些问题所困扰。
Scratchbox相比于buildroot有几方面的改进:
运行于 chroot 的环境,完全独立于主机,编译过程将基本与主机系统无关(并且scratchbox修改了一些库,使得普通用户可以chroot到编译环境中,并且多个用户可以同时使用一套Scratchbox开发套件和完全独立的用户资源);
透过qemu模拟运行或sbrsh解决中间执行文件或类似configure测试文件运行的问题;
对(chroot后)的系统进行修定,达到足以欺骗大多数软件的效果,这并不是指的让软件可以不进行改造就可以 交叉 编译,而是使软件 误认为 这就是在目标平台上编译;
不过 Scratchbox 目前还只能编译 ARM 和 x86 的代码,不能支持 buildroot 所支持的 ppc、mips等。
本文不详述每一种环境,因此各个软件都只是点到为止(虽然可以讲得更详细一些,但这些内容还是独立出来比较好一些),不过这里还是引入一个很简单的示例,根据 scratchbox 网站上的文档,安装完成后,进行简单配置就可以使用了(Debian用户的安装可以更简单,因为该站提供Deb包,直接apt-get就行了)。通过 /scratchbox/login 登入开发环境,通过sb-menu配置一个基于 ARM 的环境(其中 Select CPU-transparency method 选qemu不要先sbrsh),然后写一个 helloword.c,编译并运行之。通过ldd可以看到,在没有任可改动的情况下,顺利的生成了ARM ELF,但在 scratchbox 里却可以在X86的主机上正常的运行!
[sbox-redice: ~] > gcc -o hello hello.c
[sbox-redice: ~] > file hellohello:
ELF 32-bit LSB executable, ARM, version 1 (ARM),
for GNU/Linux 2.0.0,dynamically linked (uses shared libs),
not stripped[sbox-redice: ~] >
./hellohelo world![sbox-redice: ~] >
-
2007-10-31
Linux下的多线程编程 - [C/C++]
本文出自:http://www.china-pub.com 作者: 姚继锋
1 引言
线程(thread)技术早在60年代就被提出,但真正应用多线程到操作系统中去,是在80年代中期,solaris是这方面的佼佼者。传统的Unix也支持线程的概念,但是在一个进程(process)中只允许有一个线程,这样多线程就意味着多进程。现在,多线程技术已经被许多操作系统所支持,包括Windows/NT,当然,也包括Linux。
为什么有了进程的概念后,还要再引入线程呢?使用多线程到底有哪些... -
2007-10-17
Linux输入输出重定向以及管道 - [UNIX/LINUX]
1、标准输入与输出我们知道,执行一个shell命令行时通常会自动打开三个标准文件,即标准输入文件(stdin),通常对应终端的键盘;标准输出文件(stdout)和标准错误输出文件(stderr),这两个文件都对应终端的屏幕。进程将从标准输入文件中得到输入数据,将正常输出数据输出到标准输出文件,而将错误信息送到标准错误文件中。
我们以cat命令为例,cat命令的功能是从命令行给出的文件中读取数据,并将这些数据直接送到标准输出。若使用如下命令:
$ cat config
将会把文件config的内容依次显示到屏幕上。但是,如果cat的命令行中没有参数,它就会从标准输入中读取数据,并将其送到标准输出。例如:
$ cat
Hello world
Hello world
Bye
Bye
$
用户输入的每一行都立刻被cat命令输出到屏幕上。
另一个例子,命令sort按行读入文件正文(当命令行中没有给出文件名时,表示从标准输入读入),将其排序,并将结果送到标准输出。下面的例子是从标准输入读入一个采购单,并将其排序。
$ sort
bananas
carrots
apples //按ctrl+d后开始执行
apples
bananas
carrots
$
这时我们在屏幕上得到了已排序的采购单。直接使用标准输入/输出文件存在以下问题:
输入数据从终端输入时,用户费了半天劲输入的数据只能用一次。下次再想用这些数据时就得重新输入。而且在终端上输入时,若输入有误修改起来不是很方便。
输出到终端屏幕上的信息只能看不能动。我们无法对此输出作更多处理,如将输出作为另一命令的输入进行进一步的处理等。
为了解决上述问题,Linux系统为输入、输出的传送引入了另外两种机制,即输入/输出重定向和管道。
1.1、输入重定向
输入重定向是指把命令(或可执行程序)的标准输入重定向到指定的文件中。也就是说,输入可以不来自键盘,而来自一个指定的文件。所以说,输入重定向主要用于改变一个命令的输入源,特别是改变那些需要大量输入的输入源。
例如,命令wc统计指定文件包含的行数、单词数和字符数。如果仅在命令行上键入:
$ wc
wc将等待用户告诉它统计什么,这时shell就好象死了一样,从键盘键入的所有文本都出现在屏幕上,但并没有什么结果,直至按下<ctrl+d>,wc才将命令结果写在屏幕上。
如果给出一个文件名作为wc命令的参数,如下例所示,wc将返回该文件所包含的行数、单词数和字符数。
$ wc /etc/passwd
20 23 726 /etc/passwd //输出分别为该文件的newline数,word数和byte数及文件名
$
另一种把/etc/passwd文件内容传给wc命令的方法是重定向wc的输入,对于既可以参数文件也可以标准输入作为输入的命令,惯例是在文件的位置用-用以表明目前是利用标准输入作为输入,看下边的英文介绍。输入重定向的一般形式为:命令<文件名。可以用下面的命令把wc命令的输入重定向为/etc/passwd文件:
$ wc < /etc/passwd
20 23 726
$
另一种输入重定向称为here文档,它告诉shell当前命令的标准输入来自命令行。here文档的重定向操作符使用<<。它将一对分隔符(本例中用delim表示)之间的正文重定向输入给命令。下例将一对分隔符delim之间的正文作为wc命令的输入,统计出正文的行数、单词数和字符数。
$ wc<>this text forms the content
>of the here document,which
>continues until the end of
>text
>delim
4 17 98
在<<操作符后面,任何字符都可以作为正文开始前的分隔符,本例中使用delim作为分隔符。here文档的正文一直延续到遇见另一个分隔符为止。第二个分隔符应出现在新行的开头。这时here文档的正文(不包括开始和结束的分隔符)将重新定向送给命令wc作为它的标准输入。
由于大多数命令都以参数的形式在命令行上指定输入文件的文件名,所以输入重定向并不经常使用。尽管如此,当要使用一个不接受文件名作为输入参数的命令,而需要的输入内容又存在一个文件里时,就能用输入重定向解决问题。
1.2、输出重定向
输出重定向是指把命令(或可执行程序)的标准输出或标准错误输出重新定向到指定文件中。这样,该命令的输出就不显示在屏幕上,而是写入到指定文件中。
输出重定向比输入重定向更常用,很多情况下都可以使用这种功能。例如,如果某个命令的输出很多,在屏幕上不能完全显示,那么将输出重定向到一个文件中,然后再用文本编辑器打开这个文件,就可以查看输出信息;如果想保存一个命令的输出,也可以使用这种方法。还有,输出重定向可以用于把一个命令的输出当作另一个命令的输入(还有一种更简单的方法,就是使用管道,将在下面介绍)。
1.2.1、输出重定向的一般形式为:命令>文件名。例如:
$ ls > directory.out
$ cat directory.out
ch1.doc ch2.doc ch3.doc chimp config mail/ test/
$
将ls命令的输出保存为一个名为directory.out的文件。
注:如果>符号后边的文件已存在,那么这个文件将被重写。
1.2.2、为避免输出重定向中指定文件只能存放当前命令的输出重定向的内容,shell提供了输出重定向的一种追加手段。输出追加重定向与输出重定向的功能非常相似,区别仅在于输出追加重定向的功能是把命令(或可执行程序)的输出结果追加到指定文件的最后,而该文件原有内容不被破坏。如果要将一条命令的输出结果追加到指定文件的后面,可以使用追加重定向操作符>>。形式为:命令>>文件名。例如:
$ ls *.doc>>directory.out
$ cat directory.out
ch1.doc ch2.doc ch3.doc chimp config mail/ test/
ch1.doc ch2.doc ch3.doc
$
1.2.3、和程序的标准输出重定向一样,程序的错误输出也可以重新定向。使用符号2>(或追加符号2>>)表示对错误输出设备重定向。例如下面的命令:
$ ls /usr/tmp 2> err.file
可在屏幕上看到程序的正常输出结果,但又将程序的任何错误信息送到文件err.file中,以备将来检查用。
1.2.4、还可以使用另一个输出重定向操作符(&>)将标准输出和错误输出同时送到同一文件中。例如:
$ ls /usr/tmp &> output.file
1.2.5、利用重定向将命令组合在一起,可实现系统单个命令不能提供的新功能。例如使用下面的命令序列:
$ ls /usr/bin > /tmp/dir
$ wc –w < /tmp/dir
459
统计了/usr/bin目录下的文件个数。
2、管道
将一个程序或命令的输出作为另一个程序或命令的输入,有两种方法,一种是通过一个临时文件将两个命令或程序结合在一起,例如上个例子中的/tmp/dir文件将ls和wc命令联在一起;另一种是Linux所提供的管道功能。这种方法比前一种方法更好。
2.1、管道可以把一系列命令连接起来,这意味着第一个命令的输出会作为第二个命令的输入通过管道传给第二个命令,第二个命令的输出又会作为第三个命令的输入,以此类推。显示在屏幕上的是管道行中最后一个命令的输出(如果命令行中未使用输出重定向)。
通过使用管道符“|”来建立一个管道行。用管道重写上面的例子:
$ ls /usr/bin|wc -w
1789
再如:
$ cat sample.txt|grep "High"|wc -l
管道将cat命令(列出一个文件的内容)的输出送给grep命令。grep命令在输入里查找单词High,grep命令的输出则是所有包含单词High的行,这个输出又被送给wc命令,wc命令统计出输入中的行数。假设sample.txt文件的内容如下:
Things to do today:
Low:Go grocery shopping
High:Return movie
High:Clear level 3 in Alien vs. Predator
Medium:Pick up clothes from dry cleaner
那么该管道行的结果是2。
2.2、命令替换
命令替换和重定向有些相似,但区别在于命令替换是将一个命令的输出作为另外一个命令的参数。常用命令格式为:
command1 `command2`
其中,command2的输出将作为command1的参数,`号不是单引号,而是键盘右上角~下边的那个符号。需要注意的是这里的`符号,被它括起来的内容将作为命令执行,执行后的结果作为command1的参数。例如:
$ cd `pwd`
该命令将pwd命令列出的目录作为cd命令的参数,结果仍然是停留在当前目录下。
附:一份英语文档中的解释
Redirection and piping:The way that UNIX/Linux utilities are typically combined is via piping and redirection.Many utilities either automatically or optionally take their input from STDIN, and send their output to STDOUT (with special messages sent to STDERR). A pipe sends the STDOUT of one utility to the STDIN of another utility (or to a new invocation of the
same utility). A redirect either reads the content of a file as STDIN, or sends the STDOUT and/or STDERR output to a named file. Redirects are often used to save data for later or repeated processing (with the later utility runs using STDIN redirection).In almost all shells, piping is performed with the vertical-bar | symbol, and redirection with the greater-than and less-than symbols: > and <. To redirect STDERR, use 2>, or &> to redirect both STDOUT and STDERR to the same place. You may also use a doubled greater-than (>>) to append to the end of an existing file. For example:
$ foo fname | bar - > myout 2> myerr
Here, the utility foo probably processes the file named fname, and outputs to STDOUT. The utility bar uses a common convention of specifying a dash when output is to be taken from STDIN rather than from a named file (some other utilities take only STDIN). The STDOUT from bar is saved in myout, and its STDERR in myerr.







