Linux Tutorials

Why is Sendmail not accepting connections from any host except localhost (127.0.0.1)

2008-04-02T20:05:00.004+05:30

By default, Sendmail is configured to only accept connections from localhost (127.0.0.1). To allow connections from ALL hosts, please do the following:

Note: This will setup Sendmail to allow connections from ALL IP's and all interfaces on the machine.

Install the package with the following option # up2date sendmail-cf
Edit /etc/mail/sendmail.mc and change the following line:
DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnl
To look like the following (removing the localhost restrictions):
DAEMON_OPTIONS(`Port=smtp,Name=MTA')dnl
Save the file.
Execute the following command to rebuild the Sendmail files: make -C /etc/mail
Restart Sendmail with: service sendmail restart ...

How do you setup cyrus for POP3 to work with Sendmail with Red Hat Enterprise Linux 4?

2008-04-02T20:02:00.000+05:30

All of the command provided are required to run locally. We recommend that you back-up any files that you will edit, before editing them. For example, us the cp (copy) command to save an instance of the file before you edit:

cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc.bak
To begin, first run:

service saslauthd status
service cyrus-imapd status
service sendmail status
To setup cyrus, leave /etc/imapd.conf as default from the RPM package. For /etc/cyrus.conf leave as default except for the following: # UNIX sockets start with a slash and are put into /var/lib/imap/sockets
SERVICES {
# add or remove based on preferences
imap cmd="imapd" listen="imap" prefork=5
# imaps cmd="imapd -s" listen="imaps" prefork=1
pop3 cmd="pop3d" listen="pop3" prefork=3
# pop3s cmd="pop3d -s" listen="pop3s" prefork=1
# sieve cmd="timsieved" listen="sieve" prefork=0

# these are only necessary if receiving/exporting usenet via NNTP
# nntp cmd="nntpd" listen="nntp" prefork=3
# nntps cmd="nntpd -s" listen="nntps" prefork=1

# at least one LMTP is required for delivery
# lmtp cmd="lmtpd" listen="lmtp" prefork=0
lmtpunix cmd="lmtpd" listen="/var/lib/imap/socket/lmtp" prefork=1

# this is only necessary if using notifications
# notify cmd="notifyd" listen="/var/lib/imap/socket/notify" proto="udp" prefork=1
}

Note: You will need imap even if you are not planning on running an imap server.

Next, set the password for cyrus:

passwd cyrus
Set the saslauthd password: saslpasswd -c cyrus
su to the cyrus user and execute: /usr/lib/cyrus-imapd/mkimap
Add a user with both passwords: useradd testuser
saslpasswd -c testuser
Now restart or start the services and turn them on to start at boot time: service saslauthd restart
service cyrus-imapd restart
service sendmail restart
service xinetd restart
chkconfig saslauthd on
chkconfig cyrus-imapd on
chkconfig sendmail on
You might need to reboot to allow cyrus to take ownership of the port.

Login in as cyrus to cyrus-imap:

cyradm --user cyrus localhost
You can type help for more information. The prompt should also change: localhost.localdomain> help
Now make the users mailbox. Note the "user." must come before the username: localhost.localdomain> cm user.testuser
You can now exit: localhost.localdomain> quit
You can now test the user: telnet localhost 110
the login:
user testuser
pass test
You should see:
+OK Name is a valid mailbox
+OK Mailbox locked and ready
You now have cyrus working correctly. The mail box should show up:
/var/spool/imap/t/user/
You will now need to set Sendmail up to receive mail and send it to cyrus. Edit the configuration file /etc/mail/sendmail.mc and add this line to the bottom: dnl MAILER(smtp)dnl
dnl MAILER(procmail)dnl
define(`confLOCAL_MAILER', `cyrus')
MAILER(`cyrus')
If you want Sendmail to be able to receive mail from other places then itself, change this line from: DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnlto dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA')dnlNext, execute: # m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
You will then need to restart Sendmail: service sendmail restart
You can test, by doing: # mail testuser@localhost
Subject: test
test
.
#make sure you put the ?.? at the end of the email.
The mail should show up in: /var/spool/imap/t/user/testuser/ and you should be able to check it with any local POP3 client

How do I configure Samba to use domain accounts for authentication?

2008-04-02T19:57:00.000+05:30

The following is a bare bones smb.conf file that will authenticate users off of a domain controller. Keep in mind that if you wish to apply user/group permissions locally you should create identical user accounts on the linux system. These accounts do not need to have access, and it is advisable to set their shells to /bin/false.

Replace MACHINE with the name you wish your server to be associated with via netbios. Normally this would be identical to your hostname. Replace WORKGROUP with the name of the domain you wish to be a member of.

If the "password server = *" is used, Samba will search for a domain controller to authenticate against. Alternatively you can specify the tcp/ip address(es) of your domain controller(s).

[global]
workgroup = EXAMPLE
netbios name = MACHINE
password server = *
encrypt passwords = Yes
preferred master = No
domain master = No

How do I join a Windows 2000/2003 Active Directory domain from Red Hat Enterprise Linux?

2008-04-02T19:51:00.000+05:30

After configuring your /etc/samba/smb.conf and your /etc/krb5.conf files appropriately, enter the following command:
net ads join -U Administrator
You will then be prompted to enter the domain administrator's password. The domain name used is whatever workgroup = field is set to in smb.conf. To verify that the join is successful, look for the server in the directory controller's server management tool

How do I configure the Winbind service to authenticate itself so it is compatible with Windows domain controllers that restrict anonymous access to us

2008-04-02T19:50:00.000+05:30

The Winbind service communicates with Domain Controllers (DC's) using anonymous (non-authenticated) connections by default, to properly emulate the behavior of legacy Windows clients such as NT and Win9x when requesting user and group data from DC's.

This behavior can cause problems for Winbind if the domain uses DC's which are set up to restrict anonymous access. For example, if the Windows admin selected the "Permissions compatible with Windows 2000 servers only" option when adding the DC role to a Windows 2000 or Windows 2003 server. Although Winbind may be configured as recommended by Red Hat, the service may still fail to work properly because the DC's are refusing to service anonymous requests.

If this option was chosen during DC setup or other security policy changes have been made to the DC afterwards - perhaps even by the installation of a Windows service pack - the DC will refuse to provide the user and group information requested by Winbind if the service has not been configured to authenticate itself with a valid domain user account. As noted above, Winbind will use anonymous connections until configured to do otherwise.

The --set-auth-user option of the wbinfo command can be used to set a domain user account and password for the Winbind service to use. The user account specified needs to exist on the domain, but any regular user account should suffice -- it should not be necessary to specify a Domain Admin account unless security policies and/or user rights have been extensively modified on the DC's.

Many system administrators will create a user on the domain named winbind for the Winbind service to use, so that Winbind's activities can be monitored or audited on the DC. The example command shown below assumes such a user exists on the domain.

When running wbinfo --set-auth-user, it is not necessary to provide the password for the specified domain user on the command line. wbinfo will prompt for the specified user's password and using the command in this manner prevents the user's password from being stored in the root user's command history:

# wbinfo --set-auth-user winbind
Password:
The username and password provided will be stored for future use. Care should be taken to type the user's password correctly when prompted, because no error message will be displayed if the password is entered incorrectly.

The command can be run again whenever necessary to change the username and/or password the Winbind service should use.

wbinfo --get-auth-user can be run to view the username and password currently set for the Winbind service:

# wbinfo --get-auth-user
TESTDOMAIN+winbind%thispassword
In the example output shown above, TESTDOMAIN is the example domain's "short" domain name and the string following the percent symbol - thispassword in this example - is the password set for the winbind user.

For more information regarding Winbind and Samba configuration, we would recommend the following sources of information:

The wbinfo man page, viewable by running the command man wbinfo.
The smb.conf man page, viewable by running the command man smb.conf.
The Samba documentation contained in /usr/share/doc/samba-, on any Red Hat system with the base samba RPM installed. Complete illustrated reference manuals are provided in both PDF and HTML formats in this directory.

I made some changes to my /etc/inittab file. How can I make those changes effective without rebooting?

2008-04-02T19:48:00.002+05:30

To make changes to the /etc/inittab effective without a reboot, issue either of these two commands as the root user:
# init q

# telinit q
The init q or telinit q command wakes up init and tells it to re-examine the /etc/inittab file so changes to the file are effective immediately.

How do I change the runlevel for a system without rebooting?

2008-04-02T19:48:00.001+05:30

To change the runlevel for a system without rebooting or changing the /etc/inittab file, execute the following command as the root user:

telinit With having the following values:

0 ? Halt

1 ? Single-user mode

2 ? Not used (user-definable)

3 ? Full multi-user mode

4 ? Not used (user-definable)

5 ? Full multi-user mode (with an X-based login screen)

6 ? Reboot

How do I change the system to boot into the graphical login (runlevel 5) instead of text mode (runlevel 3)?

2008-04-02T19:47:00.001+05:30

In order to change what run level the system boots to, modify the /etc/inittab file. This is a sample from the inittab file:

# Default runlevel. The runlevels used by RHS are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:3:initdefault:
The runlevel is determined by this line:

id:3:initdefault:
Change the above line to:

id:5:initdefault:and your system will now boot to runlevel 5 or to a graphical login

How do I change the system to boot into text mode (runlevel 3) instead of the graphical login (runlevel 5)?

2008-04-02T19:42:00.000+05:30

In order to change what runlevel the system boots to, modify the /etc/inittab file. This is a sample from the inittab file:

# Default runlevel. The runlevels used by RHS are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:5:initdefault:
runlevel is determined by this line:

id:5:initdefault:
Change the above line to:

id:3:initdefault:and your system will now boot to runlevel 3 or text mode.

How do I disable X from loading on start up, but start and exit the X Window System when I need to?

2008-04-02T19:38:00.000+05:30

In order to disable X from loading during the boot sequence, it is recommended that you boot your system to runlevel 3. Booting your system to runlevel 3 will still allow you to log in as a system user but will provide a command line login instead of a graphical login. Once you are successfully logged in, you can then use the startx command to bring your system into graphical mode or runlevel 5:

startx
In order to shut down GNOME or KDE, go to your Main Menu and select the "Log Out" menu option. This should shut down Xwindows and return you to a command line or runlevel 3. In order to configure your system to boot into runlevel 3, you can modify the /etc/inittab file and change the id initdefault line to runlevel 3 as shown below:

# Default runlevel. The runlevels used by RHS are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:3:initdefault:

Why does lsmod not show the mt or SCSI card modules as being loaded?

2008-04-02T18:55:00.000+05:30

If lsmod does not show the mt or SCSI card modules as being loaded, you can load the SCSI modules by doing the following:
Determine modules SCSI card is using
Run:
# modprobe module_name
# modprobe st

The tape drive should now appear as a sequential access device and you should be able to access the tape drive now.

# cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: SONY Model: SDX-400V Rev: 0101
Type: Sequential-Access ANSI SCSI revision: 02
------------------------------------------------------------------

How do I enforce user passwords to expire after a set amount of time?

2008-04-01T19:59:00.000+05:30

To force users to change their passwords the maxdays variable has to be set for that user. An example of how to do this can be found below:

chage -M 30 The above will expire the associated users password every 30 days. This can also be done when first assigning a password to a user when creating their account with the command below: passwd -x 30
It would also be wise to warn your users that their account password is about to expire. This can be done by changing the warndays variable shown below.

chage -W 4 This will warn the user 4 days before their password expires that they will need to change their password.

To retrieve expiry information about an existing account, use the command below:

chage -l The above command will result in the output below: Minimum: 0
Maximum: 30
Warning: 4
Inactive: -1
Last Change: Mar 03, 2005
Password Expires: Apr 02, 2005
Password Inactive: Never
Account Expires: Never

Further information on all of the options for the chage and passwd commands can be foudn in the man pages. To view the manual page, issue the commands below at a terminal:

man chage
man passwd

How can password expiration be turned off?

2008-04-01T19:57:00.000+05:30

The password expiration information for a user is contained in the last 6 fields of the file /etc/shadow (the last field is reserved for future use). Password expiration for a particular user can be disabled by editing the shadow file and removing values from the corresponding colon delimited entries in the file

How do I set an expiration date for a users password or lockout out a user using User Manager?

2008-04-01T18:47:00.000+05:30

To use the User Manager, you must be running the XWindow System, have root privileges, and have the redhat-config-users RPM package installed. To start the User Manager from the desktop, go to the Main Menu Button (on the Panel) => System Settings => Users & Groups. Or, type the command redhat-config-users at a shell prompt (for example, in an XTerm or a GNOME terminal).

Click the Account Info tab. Select Enable account expiration if you want the account to expire on a certain date. Enter the date in the provided fields. Select User account is locked to lock the user account so that the user cannot log in to the system.

How do I make users set a password the first time they log in using command line tools?

2008-04-01T18:46:00.000+05:30

If a system administrator wants a user to set a password the first time the user log in, the user's initial or null password can be set to expire immediately, forcing the user to change it immediately after logging in for the first time.

To force a user to configure a password the first time the user logs in at the console, follow these steps. Note, this process does not work if the user logs in using the SSH protocol.

Lock the user's password - If the user does not exist, use the useradd command to create the user account, but do not give it a password so that it remains locked. If the password is already enabled, lock it with the command:usermod -L usernameForce immediate password expiration - Type the following command: chage -d 0 usernameThis command sets the value for the date the password was last changed to the epoch (January 1, 1970). This value forces immediate password expiration no matter what password aging policy, if any, is in place.

How do I lock out a user after a set number of login attempts in Red Hat Enterprise Linux 2.1?

2008-04-01T17:58:00.000+05:30

The PAM (Pluggable Authentication Module) module pam_tally keeps track of unsuccessful login attempts then disables user accounts when a preset limit is reached. This is often referred to as account lockout.
To lock out a user after 4 attempts, two entries need to be added in the /etc/pam.d/system-auth file:

auth required /lib/security/pam_tally.so onerr=fail no_magic_root
account required /lib/security/pam_tally.so deny=3 no_magic_root reset

In addition, run the command touch /var/log/faillog to create this empty file. This is where the attempts will be logged.

The options used above are described below:

onerr=fail
If something strange happens, such as unable to open the file, this determines how the module should react.

no_magic_root
This is used to indicate that if the module is invoked by a user with uid=0, then the counter is incremented. The sys-admin should use this for daemon-launched services, like telnet/rsh/login.

deny=3
The deny=3 option is used to deny access if tally for this user exceeds 3.

reset
The reset option instructs the module to reset count to 0 on successful entry.

See below for a complete example of implementing this type of policy:

auth required /lib/security/pam_env.so
auth required /lib/security/pam_tally.so onerr=fail no_magic_root
auth sufficient /lib/security/pam_unix.so likeauth nullok
auth required /lib/security/pam_deny.so

account required /lib/security/pam_unix.so
account required /lib/security/pam_tally.so deny=5 no_magic_root reset

password requisite /lib/security/$ISA/pam_cracklib.so retry=3
password sufficient /lib/security/$ISA/pam_unix.so nullok use_authtok md5 shadow
password required /lib/security/$ISA/pam_deny.so

session required /lib/security/$ISA/pam_limits.so
session required /lib/security/$ISA/pam_unix.so

For more detailed information on the PAM system please see the documentation contained under /usr/share/doc/pam-

How do I prevent the reuse of old passwords?

2008-04-01T17:48:00.000+05:30

Resolution:The PAM module pam_unix.so can be configured to maintain a list of old passwords for every user prohibiting the reuse of old passwords. The list is located in the /etc/security/opasswd file. This is not a plain text file, but should be protected the same as the /etc/shadow file. This is normally referred to as password history.

To remember the last 15 passwords, add the line below to the /etc/pam.d/system-auth file:

password sufficient /lib/security/pam_unix.so use_authtok md5 shadow remember=15

You can replace the number 15 used above with an integer you want, to enforce your password security policy.

How do I disable null passwords?

2008-04-01T17:24:00.001+05:30

Resolution:A null password allows users to log onto a system without having to first supply a valid password string. When users have null passwords, they can press the [Enter] key when prompted for a password and gain access to systems without a password. This poses a significant security risk to the system and to the accountability of actions performed by users.

To disable null passwords make a backup of the /etc/pam.d/system-auth file, then modify the original by removing nullok from one of the lines.
Backup /etc/pam.d/system-auth
cp /etc/pam.d/system-auth /etc/pam.d/system-auth.backup
Remove nullok from the following line in /etc/pam.d/system-auth

auth sufficient /lib/security/pam_unix.so likeauth nullok

after removal the line should look like:

auth sufficient /lib/security/pam_unix.so likeauth

Managing Unix security part 2

2008-03-31T18:14:00.001+05:30

Administrators (and, if they log onto the servers, users) have to care about security of the system. Why? Maybe you are concerned about attackers breaking in, or maybe auditors or insurers require you maintain a certain level of security. Make sure that everyone understands what the drivers are. This seems obvious but is surprisingly rare. Security staff tend to see security as intrinsically a good thing that needs no justification, so they forget to give one. But administrators tend to see security as intrinsically an annoyance which make it more difficult for them to do their jobs, so they very much need one. Of course, telling all your staff where the security weak points are in detail may be counterproductive, but an overview that includes real-life examples of security breaches can get people interested in the topic.

Having understood why security is important to your organisation, the administration staff need to know what the rules are. There are a couple of things you can be pretty certain of: Your administrators will think that some of the security rules are pointless, and they will think that some of the rules are completely unworkable. They will probably be entirely correct in some of their concerns. A rule that seems entirely workable when devised often turns out to have a major flaw in real life. For example, a rule might ban NFS, but if the company has just invested heavily in an application that requires it, security planners are going to have to give some ground.

Listen to your administrators. If they say that a security rule isn't workable, don't ignore them. Find out their reasons. Ideally, everyone should end up in agreement (if grudging) that either the rule needs amending or there is a workable way to implement it. Not involving the administrators is just asking for the rules to be bypassed, and whilst it might seem easy to blame the staff for this, the security people are at least as much at fault. In some cases, making a rule workable might require having a vendor make software changes. If the vendor is one of the big boys and you're not a major customer, this may be unrealistic (but you can try). If the vendor is a smaller company or the software is open source, you should have a lot more luck getting changes put in, or least getting a good explanation of why it can't be done that way with suggestions of alternatives.

Because of the lack of security updates, security rules that prevent unauthorised command line access are especially important, as they protect the server perimeter (not to be confused with the network perimeter). On the technical side, this means rules that prevent passwords going across accessible networks in the clear, and rules which prevent a server being tricked into a trusted relationship with another host. They cover protocols such as telnet, rsh, ftp, DNS, NIS, and NFS. They cover password rules: technical rules preventing weak passwords and enforcing password aging, for example. Most Unix/Linux flavours have other advanced account controls you can use as well, such as the ability to check passwords against a dictionary, prevent a password being reused within a time limit, and expire passwords after a period of time (normally one month).

Enforcing strong passwords is critical. Running a password cracker such as Crack or John the Ripper can be a real eye-opener: If you haven't trained your users in choosing good passwords, it's very likely that you'll crack at least a few accounts within a minute.

Ross Anderson at Cambridge University has done some research on different methods for choosing passwords. He looked at randomly chosen simple passwords to see how secure and how easy to remember (Post-it Note resistant) they were. He found one method that was easy to remember and difficult to crack. In this initial letter method, the password is made up of the initial letters of a phrase that is meaningful to the user, with some numbers and punctuation substituted for letters. For example, if the phrase was "Hit me baby one more time -- Britney Spears," the password might be *Mb1mtBs.

Encourage administrators to use the initial letter method of choosing passwords, and run regular cracking sessions to spot users who don't. A quiet notification that someone's password has been cracked, along with a reminder of how to choose a strong password and the reasons for doing so, should be enough to get the user to change his ways.

With all this security in place across a large server environment, the risk goes up that either the security or the functionality of the system will get broken by someone either making a mistake or changing something on purpose. For example, file permissions may be changed, or a configuration file might be incorrectly amended. There has to be some ongoing check for compliance, and an audit every two years won't be enough. There doesn't have to be anything too complex or expensive. A simple shell script that runs daily or weekly should be sufficient. It should check everything it can, report non-compliance, and, if at all possible, automatically correct problems to bring the server in line with the standards. A great deal of caution is appropriate when having a script make automatic corrections to a server, but the alternative -- having someone manually fix the problems -- probably will not stand the test of time.

Having everyone protect their user accounts is important, but may not be critical if there are accounts no one is looking after. These may be accounts of people who have left the organization, or they could be accounts installed by an application that has never been used. Of course you have a well-documented process for deleting accounts when someone leaves the company or moves departments, but processes are never perfect, so you should have a backup method. In a script (like the script to check ongoing compliance), check for dormant accounts that have never been used, or haven't been used for a few months. Be careful -- no one's going to thank you if the guy on call can't log onto a server at 3 a.m. because his account was automatically deleted the previous month. You can automatically delete accounts, but if you do, you have to be very confident that all your administrators and users log onto every server they need access to on a regular basis, and you need to exclude application accounts. One way of doing this is to have administrators manually run a script once a month that automatically logs onto every server they manage, just as a handshake to update the last login time (though you can use it to do other things as well, such as change passwords).

Social engineering is an obvious way for an attacker to gain command-line access to a server. Many IT departments are large and geographically spread out, maybe even with teams in different countries. Some of your administrators may never have met each other. If a new person joins the team, some people may not find out for some time. These sorts of departments are easy targets for a social engineering attack. An attacker with some minimal knowledge of how things work in the organisation has a good chance of getting an account created or a password reset with a well-placed, convincing phone call.

It may sound odd, but make sure your administrators know who is on the team. It's not uncommon these days for companies to have administration teams spread across different locations; maybe even across countries or continents. When someone gets a phone call, he needs to know who is a legitimate administrator and who isn't. It's amazing how often new staff are introduced to the people in their own office, but no one else is even told they exist. (Even introductions within the office can be missed, until a few days later someone timidly asks who the guy in the corner is and what he's doing.) Following the same principle, the security team should be familiar to the administrators (and vice versa) with easy communication routes in both directions, whether it is just being contactable on the telephone or sending out a regular security newsletter. This all seems simple, but in modern companies it can be hard to figure out which department you're in yourself after the latest reorganisation, without worrying about who is in other departments.

Recognise that keeping the servers secure is an achievement to be proud of, and the people who manage it should be shown appreciation. Keep up the security training and awareness with boosters for existing staff as well as education for new people. Keep involving the administrators and listening to them. Consider having non-security staff going on external security training courses as part of their personal development.

Be aware of security fatigue. Time goes by and there are no (known) security breaches. People naturally start dropping their guard. What was all that security business about anyway? We don't have a security problem. Before you know it, you're back to square one. There are a few strategies to counter this. Easiest of all, keep an eye on security break-ins reported in the media and pass on details to staff. This at least keeps awareness high that the criminals are out there, even if they aren't in here right now. Repeat audits are important, but do the audit correctly. There is very little point in auditing the security documentation: most attackers won't check your security rules before breaking in, they'll just see what is actually implemented. Better to do spot checks on real servers, even if only a small number. Also, run some penetration tests. Give some external consultant access inside your perimeter and have him try to break into the computers. Involving external people is useful as they might spot problems you've missed, and they should be able to give you some insight as to how you compare with other organisations.

All being well, you end up with security that is good enough to protect the business whilst being workable for the administrators and acceptable for the owners and users. All being really well, you'll still have that security in place a year or two down the line

Managing Unix security in large organisations, part 1

2008-03-31T18:09:00.000+05:30

Managing security in large organizations can be a challenge. Here are some practical tips for keeping your organization sealed tight.
In large heterogeneous Unix/Linux environments with several hundred servers, keeping up to date with security patches, which are the number one requirement for strong security, is next to impossible. Pushing out patches across hundreds of servers, with a mix of different operating systems, kernels, and versions, is complex, time-consuming, risky, and very expensive. For most large companies a six-month refresh of security patches is the best you can hope for, and even that is optimistic. This means that your servers will have exploitable security bugs most of the time. The challenge is not to eliminate those bugs but to mitigate the risk they pose.

The vast majority of exploitable bugs require local access for an attacker to be able to use the exploit; he has to be logged onto the server. We tend to think that the main aim of Unix security is to guard root, but for most servers in large companies this doesn't work. A competent attacker with any local login can probably find an exploit to get root; so the real challenge is to prevent attackers from getting a local command line login.

This is a big problem because all of your administrators (and maybe users too) have an account. Any one of them can allow an attacker in. No one can remember 200 passwords, so if an attacker gets a login to one box, he's probably got a login to tens or hundreds.

The first way to mitigate this risk is to compartmentalise the users and administrators. You may be able to prevent some people having command line access. Maybe they can be locked into a menu or launched into a single command (e.g. to change their password). In some cases, a chrooted environment may be suitable for users.

One method is to have each of your administrators responsible for certain groups of servers. If Bob only has access to 30 servers and his password is compromised, at least you've limited the damage that can be done. Unfortunately, this works less well in practise, because it clashes with a key aim of most IT departments to save money by consolidating support personnel. If you have your administrators compartmentalised into five teams, that's five different on-call rotas. Worse that this, you risk splitting up the Unix expertise and limiting communication between teams, which reduces everyone's effectiveness. This is a problem we'll see again: the contention between security and efficiency.

There are ways that compartmentalising can have a chance of working, but they are not very satisfactory. For example, separate teams could perform day-to-day administration on groups of systems, but with only one on-call rota shared between the teams. You set up a special on-call user account which is maintained across the servers, store the passwords securely, and allow the on-call engineer to access them as needed. If a password is accessed, change it the next day. In this way, any administrator can access any server.

Notice that identification and authentication are separated from authorisation in this model. Identification and authentication occur not on the server but when the password is accessed, which means that there needs to be a reasonably strong authorisation method for password access. When people access the server with this protocol, they are merely confirming that they are someone who has gained access to the password, which in itself is not too impressive. Whatever method of holding and accessing passwords is used, it is important that only approved people can access the on-call password and that the identity of the person who does so is correctly recorded.

With this route there's still a basic problem on the efficiency side. The people on call are required to fix problems, at all hours of the night, on systems that they do not normally log onto and so have no familiarity with. In the unlikely event that the servers are either very well documented or have very standarised builds, this might work. More likely, every server has its quirks and having a problem worked on by someone unfamiliar with those could add substantially to the time it takes to fix problems and get the production system working again.

There are other methods of compartmentalising, but these mostly restrict who has access to root commands (e.g. with the wheel group, user roles, or sudo). Because of all those security holes, this is rather like locking the stable door after the horse has bolted. To be fair, it's not pointless - not every attacker will have root exploits up his sleeve - but it's certainly not sufficient.

One other compartmentalisation method which is worth implementing is to ensure that users do not have write access to others' home directories. By default they should not, but it's common for someone to allow world write access for convenience. This allows attackers who break into one account to take control of others too, even before they have root access.

Whether or not you can compartmentalise your administrators and users, there are other techniques to make it more difficult for attackers to gain command-line user access.

One option is to enforce access by secure shell with public/private key pairs and pass-phrases. This puts a significant barrier in the way of an attacker. Instead of just getting a password, he now has to get a private key and pass-phrase: something I have and something I know. Of course for this to work, password access must be blocked.

In large environments, public/private key access are tough to manage. First there's the problem of key distribution. If there are 20 administrators and 200 servers, that's 4,000 individual keys to push around onto servers. If someone changes his key, the new key has to be pushed out to all the servers again. On the client side, each administrator's private key must be on each client machine the administrator might use to access servers (or on a network directory mounted onto every client machine). This may include home PCs and laptops that are easily stealable.

There are other problems with using public key access. Enforcing good pass-phrases and key aging is difficult. Revoking keys is also difficult. If an administrator leaves, his account should be locked or deleted. But because he had root access, he could have hidden his public key in someone else's account to give himself a back door onto the system, or created a fake user account. To some extent this can be checked for. For example, scripts can flag up accounts with more than one key, but this isn't easy, and is certainly hard work.

This is one situation where a public key infrastructure (PKI) is effective. Having a PKI allows keys to be revoked and can also overcome the key distribution problem. It doesn't make people choose strong pass-phrases though; and it is certainly non-trivial to implement.

One last thing to consider with the public key solution is the efficiency angle. A critical requirement for any solution is that administrators and users can get onto any server when they need to. As protocols get more complex, there are more things that can go wrong and prevent access, and a greater chance that the people using a product won't understand how it works and will break it, or compromise security by using it wrongly. For example, there are various file permissions which prevent secure shell working but do not cause telnet a problem. How confident are you that your solution will always allow the right people on while keeping the wrong people off? If you are not too confident, you may need an alternative method to access the servers.

Two possible server access methods that can be used in emergencies are TCP-wrappered telnet (so people can telnet to a server, but only from a specific other server within the same location, with the plain text password not leaving the computer room) and remote console access, again within a secure computer room.

All of these techniques are helpful in maintaining security

Comparing Linux and AIX

2008-03-31T17:58:00.001+05:30

Linux can learn valuable lessons from its elder cousins in the enterprise, the proprietary Unixes from the likes of IBM, Sun, and HP. Those operating systems, in turn, can learn some lessons from Linux. Comparing the features of the more enterprise-ready Linux distros with AIX, one of the leading proprietary Unixes, helps identify some of those lessons.
AIX was developed primarily for administrators, whereas Linux has been developed for and by hackers. Right from the start, a key goal of commercial Unixes is to make things easy for the people running them (though they don't always succeed). Only recently has this been a major factor in the Linux world. Some deficiencies can be fixed with improved tools, while others are more fundamental to the operating systems.

The benefit of proprietary hardware

AIX runs only on IBM's own hardware, based around the POWER family of processors, of which the POWER5 is the latest. (Apple's G5 chip is the baby brother of the POWER4.) Pretty much all the adapters and components that run in those servers are either made or rebadged by IBM. In the past IBM has almost given AIX away, making money from the hardware and services instead of the operating system software.

Using a single hardware architecture removes a big headache for AIX developers. There is no struggling to write device drivers for thousands of obscure devices, for a start. By controlling the hardware platform IBM can offer high-end hardware features such as hot-swap adapters and logical partitioning, not to mention servers where the firmware (equivalent of the BIOS) can be accessed through a Web browser when the server is powered off.

There is a significant price premium for this hardware, but there are great benefits too. CPU and memory are not all that matters (though IBM's latest model comes with up to 512GB of RAM, which should be enough for most people). Many companies are happy to pay more, or sacrifice speed, to improve reliability, availability, and serviceability. If an hour of downtime costs your business tens of thousands of dollars, this is a big deal.

Luckily, Linux is coming to have the best of both worlds. Those who want to take advantage of IBM's fancy hardware features can now run SUSE or Red Hat Linux on just about any server than IBM makes and, with logical partitioning, can even run Linux and AIX on the same server at the same time.

Device management

Linux has always been somewhat clumsy at device management. I often find myself trawling through dmesg and playing "guess the device" to figure out if some device is there and how it has been configured. Whether a particular piece of information about a device is available often seems a matter of luck. A variety of other commands with different syntaxes and outputs help to cobble together an overall picture of the hardware on a system.

AIX is a breath of fresh air in comparison. Devices can be queried easily through a few commands. The syntax for amending device settings is clear and consistent across all devices, and the amount of information available on each device is huge.

If new devices are added to a running system, a single command configures them all and installs device drivers where needed.

On my home PC, with a handful of disks and adapters, maybe I don't need the device information to be so easy to access and update. On an enterprise server with 150 PCI adapters and a few hundred disks, however, it becomes a lot more important to have good accurate information about exactly what and where everything is and what it is all doing.

Systems management

For new and experienced AIX administrators alike, AIX's Systems Management Interface Tool (SMIT) is a useful (and often essential) tool. Think of it as YaST2 with fewer sexy graphics but more functionality. About 80% of administration tasks on an AIX system can be done using SMIT. It's simple, easy to understand, mature, and it works. One nice feature is that it always saves the command or script it has run to a file, so you can do something once in SMIT and then script it thereafter. You can even say "don't do this for real, but log the command you would have run."

AIX also has a Web administration tool which, while slow (accessing via the bundled Windows or Linux PC client speeds it up) and occasionally buggy, is still a long way ahead of anything Linux has to offer. Want to set up ipsec? AIX has a nice wizard that makes it easy.

Linux is improving quickly with systems management, but some developers still seem to feel that if is isn't obscure and complicated, there's something wrong. That's fine for hackers, but companies want to employ administrators to run their systems, not hackers, and administrators like things to be easy, especially when they've got a few hundred systems to manage.

Installation and upgrades

Major OS upgrades are still a weak point for Linux. I've tried upgrades on a number of different Linux distros. Sometimes they work, sometimes they don't, and more often than not, I end up installing from scratch.

In comparison, AIX very rarely has a problem with upgrades, even when jumping several versions. I go into an AIX upgrade confident that it will work, and I go into a Linux upgrade with a feeling that it's 50/50.

For new installations, the picture is more balanced. AIX has few problems with new installs. If Linux has a problem, it's normally with some odd hardware -- not a problem AIX has to deal with, of course. Where AIX falls down is the lack of installation options. Only in the latest version of AIX has it been possible to specify a graphics-free installation, and the ability to choose packages at installation time is very limited.

AIX includes the Network Installation Manager (NIM), which can perform new installations, upgrades, software installation, and a number of other tasks across the network. It is easy to set up (via command line, menu, or wizard) and it works well. Similar tools exist for Linux, but right now they lack some of the functionality.

Security

The proprietary Unixes have traditionally fallen down a little on security, and AIX is no exception. From a commercial perspective it makes sense to not alienate your users, so usability has always taken precedence over security. The last thing IBM or Sun wants is businesses performing upgrades that stop their applications working correctly.

The result of this corporate caution is that a fresh install of AIX has gaping security holes. Services such as telnet, ftp, and rshd are enabled by default. Secure Shell (SSH) and TCP Wrappers aren't even installed (IBM ships both, but on a separate CD). AIX does come with some basic packet filtering, but there's no firewall on by default and it isn't easy to configure. Filesystem and swap space encryption aren't there either.

Compare this to Linux, where SSH is the default, most insecure services are disabled, a wealth of security software is shipped with almost every distro, and much effort has been put into helping users secure their systems.

AIX can be configured securely. IBM has a nice white paper that guides you through a lot of the tasks, but it isn't trivial to do, and the result is that a lot of companies don't, and tools like telnet are still a lot more common than they should be.

Managing disks and filesystems

Disk and filesystem management is an area where AIX is still well ahead of Linux. AIX doesn't have partitions or slices -- it has a logical volume manager instead. Logical volumes and volume groups are fundamentals on AIX, not add-ons.

To show how this can help, let's look at some of the things than can be done on AIX while the system is running normally, all using software.

Data can be mirrored and unmirrored online between any two disks. Want to mirror data between a local SCSI disk and a NAS-attached iSCSI disk of different sizes? No problem. A mirror copy can be broken off to create a "point-in-time" backup of how the system looked at that moment, then re-integrated later on.

Whole filesystems can be moved between disks, or spread out over different disks, all while users carry on oblivious. How about setting up a group of disks and making one a spare, so if another fails the spare automatically takes over, the data being copied over to it? That's simple too.

The OS can even be upgraded on a running system. You can create a copy of the OS disks, upgrade the copy, and then reboot from the upgraded disk; if it doesn't work, just switch back to the old one.

All of these, and more, come standard with the AIX operating system and can be done from simple command lines and menus. By contrast, even something like software mirroring on Linux is complicated in comparison to the one-line AIX commands.

Workload management

A perpetual problem with high-end computers is that they have too much computing capacity. Both Sun and IBM believe that their servers are often no more than 20% utilised. Luckily, all the major vendors have come up with solutions to help customers make effective use of the computing power they've spent so much money on.

Logical partitioning is the flavour of the day, with the ability to split servers up and have multiple instances running. Sun have extended this with its N-1 Grid Containers, effectively an advanced chrooted environment with multiple instances running under the same OS environment.

IBM have achieved a similar result in a slightly different way. With IBM's latest hardware, what looks like a separate computer can run on as little as one-tenth of a CPU, meaning that twenty instances of AIX can run on a dual-processor server. Even better, they can share Ethernet adapters and disks, so there is no need to have hundreds of adapters (though you can if you want to). You can even have your partitions talk to each other over the network adapter, using a virtual switch (with VLAN functionality) held in the server firmware. These partitions do not sit on top of an underlying OS; they run directly on the server.

Most of these functions are available for SUSE and Red Hat Linux on the POWER5 platform too, for those with generous hardware budgets.

Conclusion

Linux has come a long way in the last few years, but for high-end functionality and maturity, the likes of AIX and other high-end Unixes still have a significant edge. When it comes to security, though, Linux is ahead of the game, so the catching up is on the other side.

AIX Affinity With Linux

2008-03-31T17:49:00.001+05:30

Linux Background:
The Linux operating system has gained popularity through its close connection with
Internet computing and e-business. The operating system has gained a large share of this business because of the high number of applications that have been developed.
Linux’s initial attraction was that it was a “free” operating system, meaning that the source code was made available without charge. While the lack of a cost was an initial appeal, the real appeal is proving to be the applications that have been either developed or ported to Linux. Examples include Sendmail, Apache web server, and Samba (NT file and Print server emulator).
Every successful operating system has had a breakthrough application, the breakthrough application for Linux was (and still is) the Apache Web Server. Apache is by far the most widely used HTTP server on the Internet.
Further spurring the growth of Linux is the availability of the GNU tools. GNU is an
open source project that has developed a series of tools from compilers to text editors.These tools have been ported to Linux and are the tools of choice for many developers of Linux applications.

Why AIX Affinity with Linux?
The value is in the data and the applications. Developing applications is a costly and time-consuming process. If a company needed to move from a low end Intel based
system to a high performing IBM ^ pSeries or IBM RS/6000® system they
usually had to develop all new applications.
The first issue became how to assist companies that currently use Linux based application and need a mission critical system easily move to AIX 5L. The answer is to offer a set of integrated API’s and header files that will allow a Linux application to be recompiled to run on AIX 5L. AIX version 4.3.3 and AIX 5L version 5.0 today has many of the necessary APIs to run Linux application, with AIX 5L Version 5.1, there will be an even greater degree of compatibility between AIX and Linux.
The second issue is that applications are in a constant state of development, either
through enhancements or through fixing of bugs. Thus it is important that these
companies be able to work on their applications using familiar tools. The answer was to port key components of the GNU tool set, along with other open source tools, to AIX 5L.
GNU tools allows customers to work on existing applications, as well as develop new
applications using tools that they are familiar with. GNU tools are also the tools needed to recompile Linux applications to run on AIX 5L and AIX 4.3.3. This issue is addressed by the AIX Toolbox for Linux Applications, with GNU tools that have been recompiled for AIX as well as many other useful open source tools and utilities

AIX 4.3.3 and AIX 5L Version 5.0 already have affinity with Linux. Thus, you can
benefit today from AIX affinity with Linux with additional source compatibility available in AIX 5L Version 5.1.

When to use:

When considering how to best utilize AIX Affinity with Linux it is important to consider impacts to performance. AIX Affinity with Linux is designed to provide the best performance possible, however there are a couple of issues to consider that are outside the control of AIX Affinity with Linux that can influence performance.
The Linux application being deployed on AIX will have full access to all AIX
functionality, just like an application natively developed for AIX. AIX currently has a high level of compatibility with Linux, and with AIX 5L version 5.1, IBM plans to provide an even greater affinity between AIX and Linux. Thus, for a Linux application to take advantage of AIX it does not need to run through any additional layer or wrapper.
The question of performance is not one of the functionality of the recompiled Linux
application to take advantage of AIX and the IBM POWER architecture (and in the future the Intel Itanium architecture) but one of the performance of the compiler used to build the application. Most applications that have been developed natively for AIX use the IBM Visual Age compiler, while applications developed natively for Linux utilize the GNU compilers. Thus, you can expect to see a performance advantage for AIX applications that have been built using the IBM Visual Age compiler. At this time the IBM Visual Age compiler is not available for Linux applications.
The Application Programming Interface (API) method that AIX utilizes, provides a higher degree of integration between the application and the operating system than can be achieved using a layered or wrapper approach such as found in an Application Binary Interface (ABI) approach

When considering where to utilize AIX Affinity with Linux it is important to consider what applications you will be using for front-end and back-end. Many back-end applications such as databases are available on AIX. If the back-end application you are using is currently available natively on AIX, you should consider using that application rather than porting the Linux version to AIX. Another consideration is what applications in your portfolio are not performance sensitive, do not have a lot of computational requirements etc... that would benefit from the IBM Visual Age Compiler.
An example of how to utilize AIX Affinity with Linux technology is for front-end
applications. These are applications that are communicating with a back-end application.
Front-end applications typically have little or no areas where a compiler would make a significant performance advantage.
Thus, a company that develops its front-end applications on Linux can deploy them
across IBMs range of AIX and Linux enabled servers being it on Native Linux or AIX.

For back-end applications where performance is key, it is best to deploy an application that was developed for AIX. Most of these applications will have been developed utilizing the high performance IBM Visual Age compilers. However, there is nothing to preclude a back-end application from being developed on Linux and deployed on AIX. The performance difference will depend upon the application, and may be negligible.

perl - Practical Extraction and Report Language

2008-03-30T21:45:00.000+05:30

Perl is a language optimized for scanning arbitrary text files, extracting information from those text files, and printing reports based on that information. It's also a good language for many system management tasks. The language is intended to be practical (easy to use, efficient, complete) rather than beautiful (tiny, elegant, minimal).

Perl combines (in the author's opinion, anyway) some of the best features of C, sed, awk, and sh, so people familiar with those languages should have little difficulty with it. (Language historians will also note some vestiges of csh, Pascal, and even BASIC-PLUS.) Expression syntax corresponds closely to C expression syntax. Unlike most Unix utilities, Perl does not arbitrarily limit the size of your data--if you've got the memory, Perl can slurp in your whole file as a single string. Recursion is of unlimited depth. And the tables used by hashes (sometimes called "associative arrays") grow as necessary to prevent degraded performance. Perl can use sophisticated pattern matching techniques to scan large amounts of data quickly. Although optimized for scanning text, Perl can also deal with binary data, and can make dbm files look like hashes. Setuid Perl scripts are safer than C programs through a dataflow tracing mechanism that prevents many stupid security holes.

If you have a problem that would ordinarily use sed or awk or sh, but it exceeds their capabilities or must run a little faster, and you don't want to write the silly thing in C, then Perl may be for you. There are also translators to turn your sed and awk scripts into Perl scripts.

SYNOPSIS
perl [ -sTtuUWX ] [ -hv ] [ -V[:configvar] ] [ -cw ] [ -d[t][:debugger] ] [ -D[number/list] ] [ -pna ] [ -Fpattern ] [ -l[octal] ] [ -0[octal/hexadecimal] ] [ -Idir ] [ -m[-]module ] [ -M[-]'module...' ] [ -f ] [ -C [number/list] ] [ -P ] [ -S ] [ -x[dir] ] [ -i[extension] ] [ -e 'command' ] [ -- ] [ programfile ] [ argument ]...

If you're new to Perl, you should start with perlintro, which is a general intro for beginners and provides some background to help you navigate the rest of Perl's extensive documentation.

For ease of access, the Perl manual has been split up into several sections.

Overview
perl Perl overview (this section)
perlintro Perl introduction for beginners
perltoc Perl documentation table of contentsTutorials
perlreftut Perl references short introduction
perldsc Perl data structures intro
perllol Perl data structures: arrays of arrays perlrequick Perl regular expressions quick start
perlretut Perl regular expressions tutorial perlboot Perl OO tutorial for beginners
perltoot Perl OO tutorial, part 1
perltooc Perl OO tutorial, part 2
perlbot Perl OO tricks and examples perlstyle Perl style guide perlcheat Perl cheat sheet
perltrap Perl traps for the unwary
perldebtut Perl debugging tutorial perlfaq Perl frequently asked questions
perlfaq1 General Questions About Perl
perlfaq2 Obtaining and Learning about Perl
perlfaq3 Programming Tools
perlfaq4 Data Manipulation
perlfaq5 Files and Formats
perlfaq6 Regexes
perlfaq7 Perl Language Issues
perlfaq8 System Interaction
perlfaq9 NetworkingReference Manual
perlsyn Perl syntax
perldata Perl data structures
perlop Perl operators and precedence
perlsub Perl subroutines
perlfunc Perl built-in functions
perlopentut Perl open() tutorial
perlpacktut Perl pack() and unpack() tutorial
perlpod Perl plain old documentation
perlpodspec Perl plain old documentation format specification
perlrun Perl execution and options
perldiag Perl diagnostic messages
perllexwarn Perl warnings and their control
perldebug Perl debugging
perlvar Perl predefined variables
perlre Perl regular expressions, the rest of the story
perlrebackslash Perl regular expression backslash sequences
perlrecharclass Perl regular expression character classes
perlreref Perl regular expressions quick reference
perlref Perl references, the rest of the story
perlform Perl formats
perlobj Perl objects
perltie Perl objects hidden behind simple variables
perldbmfilter Perl DBM filters perlipc Perl interprocess communication
perlfork Perl fork() information
perlnumber Perl number semantics perlthrtut Perl threads tutorial
perlothrtut Old Perl threads tutorial perlport Perl portability guide
perllocale Perl locale support
perluniintro Perl Unicode introduction
perlunicode Perl Unicode support
perlunifaq Perl Unicode FAQ
perlunitut Perl Unicode tutorial
perlebcdic Considerations for running Perl on EBCDIC platforms perlsec Perl security perlmod Perl modules: how they work
perlmodlib Perl modules: how to write and use
perlmodstyle Perl modules: how to write modules with style
perlmodinstall Perl modules: how to install from CPAN
perlnewmod Perl modules: preparing a new module for distribution
perlpragma Perl modules: writing a user pragma perlutil utilities packaged with the Perl distribution perlcompile Perl compiler suite intro perlfilter Perl source filters perlglossary Perl GlossaryInternals and C Language Interface
perlembed Perl ways to embed perl in your C or C++ application
perldebguts Perl debugging guts and tips
perlxstut Perl XS tutorial
perlxs Perl XS application programming interface
perlclib Internal replacements for standard C library functions
perlguts Perl internal functions for those doing extensions
perlcall Perl calling conventions from C
perlreapi Perl regular expression plugin interface
perlreguts Perl regular expression engine internals perlapi Perl API listing (autogenerated)
perlintern Perl internal functions (autogenerated)
perliol C API for Perl's implementation of IO in Layers
perlapio Perl internal IO abstraction interface perlhack Perl hackers guideMiscellaneous
perlbook Perl book information
perlcommunity Perl community information
perltodo Perl things to do perldoc Look up Perl documentation in Pod format perlhist Perl history records
perldelta Perl changes since previous version
perl595delta Perl changes in version 5.9.5
perl594delta Perl changes in version 5.9.4
perl593delta Perl changes in version 5.9.3
perl592delta Perl changes in version 5.9.2
perl591delta Perl changes in version 5.9.1
perl590delta Perl changes in version 5.9.0
perl588delta Perl changes in version 5.8.8
perl587delta Perl changes in version 5.8.7
perl586delta Perl changes in version 5.8.6
perl585delta Perl changes in version 5.8.5
perl584delta Perl changes in version 5.8.4
perl583delta Perl changes in version 5.8.3
perl582delta Perl changes in version 5.8.2
perl581delta Perl changes in version 5.8.1
perl58delta Perl changes in version 5.8.0
perl573delta Perl changes in version 5.7.3
perl572delta Perl changes in version 5.7.2
perl571delta Perl changes in version 5.7.1
perl570delta Perl changes in version 5.7.0
perl561delta Perl changes in version 5.6.1
perl56delta Perl changes in version 5.6
perl5005delta Perl changes in version 5.005
perl5004delta Perl changes in version 5.004 perlartistic Perl Artistic License
perlgpl GNU General Public LicenseLanguage-Specific
perlcn Perl for Simplified Chinese (in EUC-CN)
perljp Perl for Japanese (in EUC-JP)
perlko Perl for Korean (in EUC-KR)
perltw Perl for Traditional Chinese (in Big5)Platform-Specific
perlaix Perl notes for AIX
perlamiga Perl notes for AmigaOS
perlapollo Perl notes for Apollo DomainOS
perlbeos Perl notes for BeOS
perlbs2000 Perl notes for POSIX-BC BS2000
perlce Perl notes for WinCE
perlcygwin Perl notes for Cygwin
perldgux Perl notes for DG/UX
perldos Perl notes for DOS
perlepoc Perl notes for EPOC
perlfreebsd Perl notes for FreeBSD
perlhpux Perl notes for HP-UX
perlhurd Perl notes for Hurd
perlirix Perl notes for Irix
perllinux Perl notes for Linux
perlmachten Perl notes for Power MachTen
perlmacos Perl notes for Mac OS (Classic)
perlmacosx Perl notes for Mac OS X
perlmint Perl notes for MiNT
perlmpeix Perl notes for MPE/iX
perlnetware Perl notes for NetWare
perlopenbsd Perl notes for OpenBSD
perlos2 Perl notes for OS/2
perlos390 Perl notes for OS/390
perlos400 Perl notes for OS/400
perlplan9 Perl notes for Plan 9
perlqnx Perl notes for QNX
perlriscos Perl notes for RISC OS
perlsolaris Perl notes for Solaris
perlsymbian Perl notes for Symbian
perltru64 Perl notes for Tru64
perluts Perl notes for UTS
perlvmesa Perl notes for VM/ESA
perlvms Perl notes for VMS
perlvos Perl notes for Stratus VOS
perlwin32 Perl notes for WindowsBy default, the manpages listed above are installed in the /usr/local/man/ directory.

Extensive additional documentation for Perl modules is available. The default configuration for perl will place this additional documentation in the /usr/local/lib/perl5/man directory (or else in the man subdirectory of the Perl library directory). Some of this additional documentation is distributed standard with Perl, but you'll also find documentation for third-party modules there.

You should be able to view Perl's documentation with your man(1) program by including the proper directories in the appropriate start-up files, or in the MANPATH environment variable. To find out where the configuration has installed the manpages, type:

perl -V:man.dirIf the directories have a common stem, such as /usr/local/man/man1 and /usr/local/man/man3, you need only to add that stem (/usr/local/man) to your man(1) configuration files or your MANPATH environment variable. If they do not share a stem, you'll have to add both stems.

If that doesn't work for some reason, you can still use the supplied perldoc script to view module information. You might also look into getting a replacement man program.

If something strange has gone wrong with your program and you're not sure where you should look for help, try the -w switch first. It will often point out exactly where the trouble is.

BUGS

Perl is at the mercy of your machine's definitions of various operations such as type casting, atof(), and floating-point output with sprintf().

If your stdio requires a seek or eof between reads and writes on a particular stream, so does Perl. (This doesn't apply to sysread() and syswrite().)

While none of the built-in data types have any arbitrary size limits (apart from memory size), there are still a few arbitrary limits: a given variable name may not be longer than 251 characters. Line numbers displayed by diagnostics are internally stored as short integers, so they are limited to a maximum of 65535 (higher numbers usually being affected by wraparound).

You may mail your bug reports (be sure to include full configuration information as output by the myconfig program in the perl source tree, or by perl -V ) to perlbug@perl.org . If you've succeeded in compiling perl, the perlbug script in the utils/ subdirectory can be used to help mail in a bug report

Installation of PHP on Unix systems

2008-03-30T21:41:00.000+05:30

There are several ways to install PHP for the Unix platform, either with a compile and configure process, or through various pre-packaged methods. This documentation is mainly focused around the process of compiling and configuring PHP. Many Unix like systems have some sort of package installation system. This can assist in setting up a standard configuration, but if you need to have a different set of features (such as a secure server, or a different database driver), you may need to build PHP and/or your web server. If you are unfamiliar with building and compiling your own software, it is worth checking to see whether somebody has already built a packaged version of PHP with the features you need.

Prerequisite knowledge and software for compiling:

Basic Unix skills (being able to operate "make" and a C compiler)
An ANSI C compiler
flex: Version 2.5.4
bison: Version 1.28 (preferred), 1.35, or 1.75
A web server
Any module specific components (such as gd, pdf libs, etc.)

The initial PHP setup and configuration process is controlled by the use of the command line options of the configure script. You could get a list of all available options along with short explanations running ./configure --help. Our manual documents the different options separately. You will find the core options in the appendix, while the different extension specific options are descibed on the reference pages.

When PHP is configured, you are ready to build the module and/or executables. The command make should take care of this. If it fails and you can't figure out why, see the Problems section.

Apache 1.3.x on Unix systems
This section contains notes and hints specific to Apache installs of PHP on Unix platforms. We also have instructions and notes for Apache 2 on a separate page.

You can select arguments to add to the configure on line 10 below from the list of core configure options and from extension specific options described at the respective places in the manual. The version numbers have been omitted here, to ensure the instructions are not incorrect. You will need to replace the 'xxx' here with the correct values from your files.

Example#1 Installation Instructions (Apache Shared Module Version) for PHP

1. gunzip apache_xxx.tar.gz
2. tar -xvf apache_xxx.tar
3. gunzip php-xxx.tar.gz
4. tar -xvf php-xxx.tar
5. cd apache_xxx
6. ./configure --prefix=/www --enable-module=so
7. make
8. make install
9. cd ../php-xxx

10. Now, configure your PHP. This is where you customize your PHP
with various options, like which extensions will be enabled. Do a
./configure --help for a list of available options. In our example
we'll do a simple configure with Apache 1 and MySQL support. Your
path to apxs may differ from our example.

./configure --with-mysql --with-apxs=/www/bin/apxs

11. make
12. make install

If you decide to change your configure options after installation,
you only need to repeat the last three steps. You only need to
restart apache for the new module to take effect. A recompile of
Apache is not needed.

Note that unless told otherwise, 'make install' will also install PEAR,
various PHP tools such as phpize, install the PHP CLI, and more.

13. Setup your php.ini file:

cp php.ini-dist /usr/local/lib/php.ini

You may edit your .ini file to set PHP options. If you prefer your
php.ini in another location, use --with-config-file-path=/some/path in
step 10.

If you instead choose php.ini-recommended, be certain to read the list
of changes within, as they affect how PHP behaves.

14. Edit your httpd.conf to load the PHP module. The path on the right hand
side of the LoadModule statement must point to the path of the PHP
module on your system. The make install from above may have already
added this for you, but be sure to check.

For PHP 4:

LoadModule php4_module libexec/libphp4.so

For PHP 5:

LoadModule php5_module libexec/libphp5.so

15. And in the AddModule section of httpd.conf, somewhere under the
ClearModuleList, add this:

For PHP 4:

AddModule mod_php4.c

For PHP 5:

AddModule mod_php5.c

16. Tell Apache to parse certain extensions as PHP. For example,
let's have Apache parse the .php extension as PHP. You could
have any extension(s) parse as PHP by simply adding more, with
each separated by a space. We'll add .phtml to demonstrate.

AddType application/x-httpd-php .php .phtml

It's also common to setup the .phps extension to show highlighted PHP
source, this can be done with:

AddType application/x-httpd-php-source .phps

17. Use your normal procedure for starting the Apache server. (You must
stop and restart the server, not just cause the server to reload by
using a HUP or USR1 signal.)

Alternatively, to install PHP as a static object:

Example#2 Installation Instructions (Static Module Installation for Apache) for PHP

1. gunzip -c apache_1.3.x.tar.gz | tar xf -
2. cd apache_1.3.x
3. ./configure
4. cd ..

5. gunzip -c php-5.x.y.tar.gz | tar xf -
6. cd php-5.x.y
7. ./configure --with-mysql --with-apache=../apache_1.3.x
8. make
9. make install

10. cd ../apache_1.3.x

11. ./configure --prefix=/www --activate-module=src/modules/php5/libphp5.a
(The above line is correct! Yes, we know libphp5.a does not exist at this
stage. It isn't supposed to. It will be created.)

12. make
(you should now have an httpd binary which you can copy to your Apache bin dir if
it is your first install then you need to "make install" as well)

13. cd ../php-5.x.y
14. cp php.ini-dist /usr/local/lib/php.ini

15. You can edit /usr/local/lib/php.ini file to set PHP options.
Edit your httpd.conf or srm.conf file and add:
AddType application/x-httpd-php .php

Note: Replace php-5 by php-4 and php5 by php4 in PHP 4.

Depending on your Apache install and Unix variant, there are many possible ways to stop and restart the server. Below are some typical lines used in restarting the server, for different apache/unix installations. You should replace /path/to/ with the path to these applications on your systems.

Example#3 Example commands for restarting Apache

1. Several Linux and SysV variants:
/etc/rc.d/init.d/httpd restart

2. Using apachectl scripts:
/path/to/apachectl stop
/path/to/apachectl start

3. httpdctl and httpsdctl (Using OpenSSL), similar to apachectl:
/path/to/httpsdctl stop
/path/to/httpsdctl start

4. Using mod_ssl, or another SSL server, you may want to manually
stop and start:
/path/to/apachectl stop
/path/to/apachectl startssl

The locations of the apachectl and http(s)dctl binaries often vary. If your system has locate or whereis or which commands, these can assist you in finding your server control programs.

Different examples of compiling PHP for apache are as follows:

./configure --with-apxs --with-pgsql

This will create a libphp5.so (or libphp4.so in PHP 4) shared library that is loaded into Apache using a LoadModule line in Apache's httpd.conf file. The PostgreSQL support is embedded into this library.

./configure --with-apxs --with-pgsql=shared

This will create a libphp4.so shared library for Apache, but it will also create a pgsql.so shared library that is loaded into PHP either by using the extension directive in php.ini file or by loading it explicitly in a script using the dl() function.

./configure --with-apache=/path/to/apache_source --with-pgsql

This will create a libmodphp5.a library, a mod_php5.c and some accompanying files and copy this into the src/modules/php5 directory in the Apache source tree. Then you compile Apache using --activate-module=src/modules/php5/libphp5.a and the Apache build system will create libphp5.a and link it statically into the httpd binary (replace php5 by php4 in PHP 4). The PostgreSQL support is included directly into this httpd binary, so the final result here is a single httpd binary that includes all of Apache and all of PHP.

./configure --with-apache=/path/to/apache_source --with-pgsql=shared

Same as before, except instead of including PostgreSQL support directly into the final httpd you will get a pgsql.so shared library that you can load into PHP from either the php.ini file or directly using dl().

When choosing to build PHP in different ways, you should consider the advantages and drawbacks of each method. Building as a shared object will mean that you can compile apache separately, and don't have to recompile everything as you add to, or change, PHP. Building PHP into apache (static method) means that PHP will load and run faster.

PHP

2008-03-30T21:38:00.000+05:30

What is PHP?

PHP is a widely-used general-purpose scripting language that is especially suited for Web development and can be embedded into HTML

Simple answer, but what does that mean? An example:

Example#1 An introductory example

Example

echo "Hi, I'm a PHP script!";
?>

Notice how this is different from a script written in other languages like Perl or C -- instead of writing a program with lots of commands to output HTML, you write an HTML script with some embedded code to do something (in this case, output some text). The PHP code is enclosed in special start and end tags that allow you to jump into and out of "PHP mode".

What distinguishes PHP from something like client-side JavaScript is that the code is executed on the server. If you were to have a script similar to the above on your server, the client would receive the results of running that script, with no way of determining what the underlying code may be. You can even configure your web server to process all your HTML files with PHP, and then there's really no way that users can tell what you have up your sleeve.

The best things in using PHP are that it is extremely simple for a newcomer, but offers many advanced features for a professional programmer. Don't be afraid reading the long list of PHP's features. You can jump in, in a short time, and start writing simple scripts in a few hours.

Installing SquirrelMail on Unix and Linux systems

2008-03-30T21:32:00.000+05:30

This chapter covers installation of SquirrelMail on generic Unix or Linux system. It does not cover installation of operating system or tools required to install web server or PHP.

Any version numbers used in examples are specific to the time when this documentation is written. If current version numbers differ, make sure that you are not using old, obsolete or vulnerable software.

Guide uses UW IMAP server as example. This IMAP server can be used in generic email setup when incoming mail is stored in /var/spool/mail directory. If you are planning to use webmail with big number of users or with bigger mailboxes, consider using different IMAP server and redesign entire email system.

Download required software
You will need:

Apache - http://httpd.apache.org/download.cgi
PHP - http://php.net/downloads.php
UW IMAP - http://www.washington.edu/imap/
SquirrelMail - http://squirrelmail.org/download.php

# install -d /usr/local/src/downloads
# cd /usr/local/src/downloads
# wget http://some-apache-mirror-server/apache/httpd/httpd-2.0.54.tar.gz
# wget http://some-php-mirror-server/get/php-4.3.11.tar.bz2/from/this/mirror
# wget ftp://ftp.cac.washington.edu/mail/imap.tar.Z
# wget http://some-sourceforge-mirror/some-path/squirrelmail-1.4.5.tar.bz2

Unpack and install apache

# cd /usr/local/src
# tar -xzvf /usr/local/src/downloads/httpd-2.0.54.tar.gz
# cd httpd-2.0.54
# ./configure --prefix=/usr/local/apache --enable-module=so
# make
# make install

Unpack and install php

# cd /usr/local/src
# tar --bzip2 -xvf /usr/local/src/downloads/php-4.3.11.tar.bz2
# cd php-4.3.11
# ./configure --prefix=/usr/local/php \
> --with-apxs2=/usr/local/apache/bin/apxs
# make
# make install

If you configure PHP compilation with --disable-all option, you must add --enable-session and --with-pcre-regex options.

Add PHP support to apache

AddType application/x-httpd-php .php

Restart apache and check if php is working

/usr/local/apache/bin/apachectl graceful

Unpack and install imap server
Unpack UW IMAP archive

# cd /usr/local/src
# tar -xzvf /usr/local/src/downloads/imap.tar.Z

Compile UW IMAP

cd /usr/local/src/imap-
make port-name EXTRADRIVERS='' SSLTYPE=unix

Replace port-name with name that matches your system. Check Makefile for possible values. If you haven't installed OpenSSL libraries and headers, use SSLTYPE=none instead of SSLTYPE=unix.

Install IMAP server binary

strip imapd/imapd
install -d /usr/local/libexec/
cp imapd/imapd /usr/local/libexec/

Enable IMAP server in inetd.conf
imap2 stream tcp nowait root /usr/sbin/tcpd /usr/local/libexec/imapd

Restart inetd

Prepare SquirrelMail directories

# mkdir /usr/local/squirrelmail
# cd /usr/local/squirrelmail
# mkdir data temp
# chgrp nogroup data temp
# chmod 0730 data temp

Unpack SquirrelMail

# cd /usr/local/squirrelmail
# tar --bzip2 -xvf /usr/local/src/downloads/squirrelmail-1.4.5.tar.bz2
# mv squirrelmail-1.4.5 www

Configure SquirrelMail
Start SquirrelMail configuration utility. Configure SquirrelMail with UW preset. Set data and attachment directories.

Configure access to SquirrelMail in Apache
Modify httpd.conf

Alias /squirrelmail /usr/local/squirrelmail/www

Options Indexes
AllowOverride none
DirectoryIndex index.php
Order allow,deny
allow from all

Log into SquirrelMail
After you add alias to SquirrelMail in apache configuration and restart apache, you should be able to access SquirrelMail by going to http://your-server/squirrelmail

SquirrelMail

2008-03-30T21:28:00.000+05:30

SquirrelMail is a standards-based webmail package written in PHP. It includes built-in pure PHP support for the IMAP and SMTP protocols, and all pages render in pure HTML 4.0 (with no JavaScript required) for maximum compatibility across browsers. It has few requirements and is easy to configure and install. SquirrelMail has all the functionality you would want from an email client, including strong MIME support, address books, and folder manipulation.

This manual supports SquirrelMail 1.4.0 and up. The 1.2.x series has been obsoleted, and is only referenced in the upgrading notes of this manual.

There are only two requirements for SquirrelMail:

A web server with PHP installed. PHP needs to be at least 4.1.0.
Access to an IMAP server which supports IMAP 4 rev 1.

It doesn't really matter what OS or web server you use, as long as the combination thereof supports PHP in a stable way. Read the instructions and suggestions in the PHP documentation to see what they recommend.

If you're building your mail system from scratch, it might be a good idea to install and test all components one by one. If you install everything at once and things don't work, the troubleshoting will be more complex. If the web server doesn't work there's not much point in trying to install PHP, for instance. Make sure that everything is working before trying to install SquirrelMail.

Choosing an IMAP server
You don't actually have to run an IMAP server yourself, but you need to be able to connect to one for SquirrelMail to work. Since IMAP is an open standard, all IMAP products should be able to communicate with each other. SquirrelMail requires that the server supports IMAP 4 rev 1, but that's the only requirement there is.

Some IMAP servers support various extensions, which are developed as a complement to IMAP. Those extensions aren't required by SquirrelMail, but many of them are supported. It's recommended to have an IMAP server that supports SORT and THREAD if possible. The SORT extension allows for server side sorting, which is a lot more efficient than having to rely on PHP for sorting. This will improve SquirrelMail's performance. If the server doesn't support the THREAD extension, SquirrelMail can't show mail conversation as threads.

If possible, the IMAP server should support Unicode. Without it some translations might be unable to use sorting and threading. Courier IMAP must be compiled with the --enable-unicode option to have Unicode support.

SquirrelMail doesn't care about how the server stores the mails, but it's generally a good idea not to have an IMAP server that store mails in the mailbox (mbox) format. Mailbox performance is low when there are many mails in the same folder and it doesn't allow both mails and subfolders at the same time in the same folder.

Another good idea is to have an IMAP server that allows the use of virtual accounts. Virtual users don't have to be system users, which usually is a good thing. Again, this is not a SquirrelMail requirement, but something that you might want to consider when choosing an IMAP server.

Some systems are delivered with an IMAP server, but if it doesn't measure up to the suggestions above, you might want to replace it. There are plenty of IMAP servers at the market, so it might be difficult to decide which one to choose. It is also difficult to recommend something, since every organization has unique demands. The IMAP Connection has a searchable database of IMAP servers, as well as more information about IMAP, but that list may not cover the entire market. There are also several sites offering advice and opinions on this matter. Read them, but make your own decision since the information at some of those sites might be outdated or biased. Remember that some of the open source alternatives are well matured products that can compete, and even surpass, the commercial servers.

Configuring PHP

Without the PHP gettext extension you lose in performance.
The PHP mbstring extension is required for translations that use multibyte or character sets but ISO-8859-1. Without the PHP mbstring extension the interface will remain usable, but some internationalization features and fixes won't be enabled. It's a must if you want to read and write Japanese emails, and users who whish to do that must also set their language option to Japanese.
The PHP XML extension is required if the DIGEST-MD5 authentication is used.

1.2 Optional server programs

Perl. SquirrelMail is shipped with some Perl scripts. One of the most useful is config/conf.pl, which will help you configure your SquirrelMail installation.
An SQL database supported by the PEAR DB library, and the PEAR DB library itself. See Using database backends for more information.
Aspell or Ispell to be able to use the SquirrelSpell plugin.

These are not a must have, since SquirrelMail will function without them, but they are adding to the experience so you might want to consider them.

Directory layout in squirrelMail

SquirrelMail files are split into subdirectories according to file type and provided functions.

squirrelmail/
class/
config/
contrib/
data/
doc/
functions/
decode/
encode/
help/
images/
locale/
plugins/
po/
src/
templates/
themes/
css/

class directory stores various classes used with mime messages, email delivery, localizations and other interface functions.

config directory stores SquirrelMail configuration files and configuration utility. conf.pl script is a perl based utility used to manage SquirrelMail configuration. The config_default.php file stores default configuration values. The config.php file stores current configuration. The file config_local.php can store local site configuration overrides and configuration options that are not supported by configuration utility. default_pref stores default user preferences that are used when a new user logs in for the first time. default_pref file was stored in the data directory before SquirrelMail 1.5.1.

The contrib directory stores files that provide extra features to SquirrelMail package, but are not used directly in the webmail interface.

The data directory is default location for SquirrelMail users' preference files. You should move that directory outside of web tree or make sure that it can't be accessed by external users. This directory is not packaged anymore since SquirrelMail 1.5.1.

The doc directory stores some documentation about SquirrelMail.

functions directory stores SquirrelMail function files. The decode subdirectory stores charset decoding functions that are used to read emails encoded in different charsets. The encode directory stores charset encoding functions that are used to convert emails to charset used in interface when user replies or forwards email written in different charset.

Under help are SquirrelMail help files. Information from these files is displayed when a user clicks on Help link in SquirrelMail menu line. Help files use XML formating. They can be translated into different languages.

The images directory stores various image files that can be used in interface.

The locale directory stores SquirrelMail translations. A user can select their preferred translation in SquirrelMail Display Options.

The plugins directory stores plugins that can be used to extend SquirrelMail functionality. Activation of plugins is controlled through the SquirrelMail configuration utility. Some plugins might also use their own configuration files or functions provided by other plugins. See README and INSTALL files in each plugin's directory.

The po directory stores scripts that are used to work with SquirrelMail translation files. xgetpo script extracts translatable strings from SquirrelMail script. mergepo script combines default strings with selected translation. compilepo script compiles selected translation. These scripts are usually used only by SquirrelMail translators.

The src directory stores scripts that are used when user accesses the webmail interface.

The templates directory stores template files that can be used in SquirrelMail 1.5.1 and later versions.

The themes directory stores SquirrelMail colour themes, and the css subdirectory stores style sheet files available to end user.

User data storage

SquirrelMail stores users' preferences and address books in simple text files. The location of these files is set with the data directory setting in the SquirrelMail configuration. SquirrelMail can also use a database or some other storage facility (if the required backend is provided by a plugin) for managing user preferences.

Users' preferences are stored in .pref files. Address books are stored in .abook files. .sig and .si files store users' signatures. Some plugins might use other files to store users' data.

When the number of files in the data directory becomes somewhat large, directory access time can be affected. In such cases, the administrator can split preference files into subdirectories by enabling directory hashing in the SquirrelMail configuration.

Configuration utility

SquirrelMail can be configured with conf.pl, a Perl script that is stored in the config/ directory. You can start it by running the configure script in the SquirrelMail base directory or by running the conf.pl script in the config directory.

# cd /path/to/squirrelmail
# cd config
# ./conf.pl

This configuration utility provides menu based configuration options:

SquirrelMail Configuration : Read: config_default.php (1.4.0)
---------------------------------------------------------
Main Menu --
1. Organization Preferences
2. Server Settings
3. Folder Defaults
4. General Options
5. Themes
6. Address Books
7. Message of the Day (MOTD)
8. Plugins
9. Database
10. Languages

D. Set pre-defined settings for specific IMAP servers

C Turn color on
S Save data
Q Quit

Command >>

Menu is controlled by entering numbers or letters that are listed on the left side.

The address book format

By default SquirrelMail stores address books in files, one per address book, named [user account name].abook. These address book files are kept in the data directory. Address books can also be stored in a database or, if the required functions are provided by a plugin, another storage facility. SquirrelMail can also be configured to lookup addresses in LDAP directories, if the PHP installation contains LDAP support.

An address book file contains five fields, which are delimited by the vertical line (|): the first field stores nicknames, short names that are used to identify address book entries; the second field stores names; the third field stores surnames; the forth field stores mail addresses; and the fifth field stores additional information.

Additional address book fields and functions can be provided by the experimental vcard address book format and some address book plugins. You can find list of address book plugins at the SquirrelMail site.

squint

2008-03-30T21:10:00.000+05:30

squint is a Squid proxy log analyzer that generates a detailed report of who is spending the most time and resources browsing the Internet. The top offenders in terms of data transfer, number of files transferred, and on-line time are reported

squint is useful for discovering problems with internet usage patterns. To determine on-line time, it is guesstimated that after a "hit" the person is reading the page for the following two minutes. So the measurement of on-line time is unreliable, but the system does provide a warning that investigation may be warranted

What is 'Content Filtering'?

2008-03-30T21:07:00.001+05:30

Your normal web filter such as Cyber Patrol, squidGuard, Net Nanny, etc, has a very large list of bad sites. If you try to go to these sites you will get blocked. I.e. your web access is filtered by web address.

The web is a fast changing place and even large web search engines such as Google or Altavista or Yahoo don't even know of half of it. This makes filtering by web address (URL) difficult as sites change and new ones come up all the time. It is impossible to have comprehensive filtering using just URLs. What is needed is something to check every page you (or your children) ever access for 'bad' subjects such as drugs, profanities, hate, pornography, etc, and disallow it if it's not suitable. This is called 'Content Filtering'.

This is why you need DansGuardian as it makes the web a cleaner, safer, place for you and your children.

As a side effect, DansGuardian also helps maintain freedom of speech by moving the censoring to the choice of the individual rather than imposing a specific ideal on the whole world.

DansGuardian

2008-03-30T21:01:00.000+05:30

DansGuardian is an award winning web content filtering proxy(1) for Linux, FreeBSD, OpenBSD, NetBSD, Mac OS X, HP-UX, and Solaris that uses Squid(2) to do all the fetching. It filters using multiple methods. These methods include URL and domain filtering, content phrase filtering, PICS filtering, MIME filtering, file extension filtering, POST limiting.
The content phrase filtering will check for pages that contain profanities and phrases often associated with pornography and other undesirable content. The POST filtering allows you to block or limit web upload. The URL and domain filtering is able to handle huge lists and is significantly faster than squidGuard.
The filtering has configurable domain, user and source ip exception lists. SSL Tunneling is supported.
The configurable logging produces a log in an easy to read format which has the option to only log the text-based pages, thus significantly reducing redundant information such as every image on a page.
Pretty much all parts of DansGuardian are configurable thus giving the end administrator user total control over what is filtered and not some third-party company.
(1) Technically DansGuardian is more of a filtering pass-through than a true proxy - but don't let that worry you!

(2) DansGuardian should work with any proxy, not just Squid. For example, it is known to work with Oops

The main features of DansGuardian are as follows:

*Significantly cheaper than IGear (one of the best commercial filters).
*Can block adverts by the use of an advert URL block list.
*Can filter text and HTML pages for obscene (sexual, racial, violent, etc) content.
*Uses an advanced phrase weighting system to reduce over or under blocking.
*Can filter sites using the PICS labeling system.
*Can filter according to MIME type and file extension.
*Can filter according to URLs including Regular Expression URLs.
*URL filtering is compatible with squidGuard black lists.
*The URL filtering is able to filter https requests.
*Can work in a 'whitelist' mode where all sites except those listed are blocked.
*Can block all IP based URLs.
*Is able to block sites when users try using the IP address of the site instead.
*Produces a log in a very human readable format.
*Optionally produces a log in CSV format for easy import into databases etc.
*Is able to log the username using either Ident or basic proxy authentication.
*It has the ability to switch off filtering for specified sites, parts of sites, browser IPs and usernames.
*Can block specified source IPs and usernames.
*Can block or limit web uploading (e.g. attachments in Hotmail).
*Has the ability to work in a stealth mode where it logs sites that would have been blocked, but does not block them. This allows you to monitor your users without them knowing.
*Uses a very intelligent algorithm to match phrases in web pages mixed in with HTML code and white space.
*Big5, Unicode and top-bit set characters can be used in search phrases.
*URL filtering is significantly faster than squidGuard.
*The configuration lists use the same incredibly fast code that allows them all to be hundreds of thousands of entries long.
*100% C++ and can compile on GCC 3.
*Can be made to re-read config files with a HUP signal.
*Works perfectly in conjunction with Squid and Oops. Also see this important information.
*Has no 3rd party library requirements (no nb++ as was used in version 1) so can be installed much easier and so is also provided as an RPM.
*Supports (adds) the squid X-Forwarded-For header line.
*Supports compressed (Content-Encoding gzip and deflate) HTML.
*Can be made to only listen on 1 IP.

Linux Commands

2008-03-29T22:06:00.000+05:30

UNIX Commands

Login and Exit
yppasswd Change password
rlogin machine Log into a remote machine
telnet machine Log into a remote machine
exit End a shell session
Ctrl-D End shell session
logout Log out of remote session

Help

man command Describes the command
man -k keyword Search for keyword

Display directory listing

ls Display directory listing
ls -l Display access permissions
ls -a Display hidden files
ls -d Display directory
ls -t Display files sorted by time
ls dir Display contents of directory
ls file Display file
ls ???* Display all files with more than 3 characters in name

Change, Create, Remove Directories

pwd Display current directory
cd Change to your home directory
cd .. Change to parent directory
cd dir Change to another directory
mkdir dir Create a new directory
rmdir dir Remove a directory (must be empty)
rm -r dir Remove a directory and everything within it

Create, Copy, Move, Delete Files

touch file Create an empty file
cp src-file dst-file Copy a file to another file
cp src-file dst-dir Copy a file to a directory
cp -r * dst-dir Copy all files and sub-directories to another directory
mv src-file dst-file Move a file to another file (renames file)
mv src-file dst-dir Move a file to another directory
mv src-dir dst-dir Move a directory to another directory
Note: filenames can be up to 255 characters long but should not contain special
characters, e.g., $ * [ ] & < >

Change file permissions

chmod o+r file Change file to allow read access to anyone
chown username file Change onwer of file to username
chgrp new-grp file Change group owner of file to new-grp
Note: -rwxr-x--- shows permissions on a regular file to be:
owner=read,write,execute; group=read,execute; others=nothing

Display Contents of a File

cat file Display contents of file
cat -v file Display non-printing characters
head file Display first 10 lines
tail file Display last 10 lines
more file Display file one screen at a time
wc file Count number of words in file
Sort, Compare, Convert, Compress Files
sort file1 > file2 Sort lines in file1 and output to file2
sort -1 -n file1 > file2 Sort by column 1 numerically and output to file2
uniq input-file Remove or report adjacent duplicate lines
unix2dos unx-file dos-file Convert from UNIX to DOS format
dos2unix dos-file unx-file Convert from DOS to UNIX format
cmp file1 file2 Compare byte-by-byte 2 files
diff file1 file2 Compare line by line 2 files
sdiff file1 file2 Compare 2 files by displaying them side by side
compress file Compress a file
gunzip file Uncompress a file
tar cf file.tar . Create a tar archive of current directory
tar xf file.tar Restore tar archive to current directory

Search for Strings within Files

grep string file(s) Display lines containing string in file(s)
grep -v string file(s) Display lines that do not contain string

Printing
lp file Print file to default printer
lpr file Print file to default printer
lpstat Check status of printer queue
lpstat -p Check status of all printer queues
lpq Check contents printer queue
lprm job# Remove job from printer queue
lprm - Remove all your jobs from printer queue
lp -Pprinter file Print to a specific printer queue
ls -l | lp Print directory listing
head file | lp Print first 10 lines of file
Redirect Output
cmd | lp Direct cmd output to printer
cmd > file Direct cmd output to file
cmd | tee file Direct output to screen and file
cmd >> file Direct cmd output to end of file
cmd1 | cmd2 Direct cmd1 output to input of cmd2
Conditional Execution
cmd1 && cmd2 Execute cmd2 only if cmd1 is successful
cmd1 || cmd2 Execute cmd2 only if cmd1 fails

BACKGROUND PROCESSES
Command Effect

cmd& Run job in background
jobs List jobs running in background in current shell session
fg Bring most recent background job into foreground
fg job# Bring job# into foreground
bg Put job into background
bg job# Put job# into background
Ctrl-z Suspend current job and put it in background (resume with fg or bg)
at timespec cmd
Ctrl-d Execute commands at a specified time
batch cmd Ctrl-d Execute commands in the batch queue
atq Display jobs running in at and batch queues
atrm job# Remove job from at or batch queue

Checking Processes

ps Display current shell processes
ps -ef Display all processes
ps -ef | grep username Display all processes for a user
kill -9 PID Stop a process
Disk Space, User and Environment Information
du -sk Display disk space of current directory in 1024k blocks
du -sk dir Display disk space of dir in 1024k blocks
df -k Display data for mounted file systems
env Display environment variable settings
setenv ENV-VAR XXX Set an environment variable
alias Display all defined aliases
umask Display current default file protection mask
whoami Display your username
id Display your user and group ids
users Display users logged in
groups Display which groups you belong to
w Display who is logged in and what they are doing

RAID

2008-03-29T21:58:00.000+05:30

1. What is RAID

* The basic idea behind RAID is to combine multiple small, inexpensive
disk drives into an array to accomplish performance or redundancy goals not
attainable with one large and expensive drive.

* This array of drives will appear to the computer as a single logical storage
unit or drive.

* RAID is a method in which information is spread across several disks, using
techniques such as :

* Disk Striping (RAID Level 0) [no redundancy,no FT]
* Disk Mirroring (RAID level 1) [redundancy,with FT]
* disk striping with parity on single disk (RAID Level 4) Not Supported
* disk striping with parity across disks (RAID Level 5)
* Linear RAID

to achieve redundancy, lower latency and/or increase bandwidth for reading or
writing to disks, and maximize the ability to recover from hard disk crashes.

* The underlying concept of RAID is that data may be distributed across each
drive in the array in a consistent manner.

* To do this, the data must first be broken into consistently-sized chunks
or strips (often 32K or 64K in size, although different sizes can be used).

* Each chunk is then written to a hard drive in RAID according to the RAID level
used.

* When the data is to be read, the process is reversed, giving the illusion
that multiple drives are actually one large drive.

2. Who Should Use RAID

* Anyone who needs to keep large quantities of data on hand
(such as a sysadmin) would benefit by using RAID technology.

* The Primary reasons to use RAID include:

* Enhanced speed

* Increased storage capacity using a single virtual disk

* Lessened impact of a disk failure

3. Hardware RAID versus Software RAID

* There are two possible RAID approaches: Hardware RAID and Software RAID.

Hardware RAID

* H/W systems manages the RAID subsystem independently from
the host and presents to the host only a single disk per RAID array.

* An example of a Hardware RAID device would be one that connects to a SCSI
controller and presents the RAID arrays as a single SCSI drive.

* An external RAID system moves all RAID handling "intelligence" into a
controller located in the external disk subsystem.

* The whole subsystem is connected to the host via a normal SCSI controller and
appears to the host as a single disk.

* RAID controllers also come in the form of cards that act like a SCSI
controller to the operating system but handle all of the actual drive
communications themselves.

* In these cases, you plug the drives into the RAID controller just like you
would a SCSI controller, but then you add them to the RAID controller's
configuration, and the operating system never knows the difference.

* Many controllers have their own BIOS and can be configured independantly
of the host computer to which they are attached, just like you use the CMOS
to configure your system.

Software RAID

* Software RAID implements the various RAID levels in the kernel disk
(block device) code. It offers the cheapest possible solution, as expensive
disk controller cards or hot-swap chassis are not required.

* Software RAID also works with cheaper IDE disks as well as SCSI disks.
With today's fast CPUs, Software RAID performance can excel against Hardware
RAID.

* The MD driver in the Linux kernel is an example of a RAID solution that is
completely hardware independent. The performance of a software-based array is
dependent on the server CPU performance and load.

Software RAID has to offer some important features:

* Threaded rebuild process
* Kernel-based configuration
* Portability of arrays between Linux machines without reconstruction
* Backgrounded array reconstruction using idle system resources
* Hot-swappable drive support
* Automatic CPU detection to take advantage of certain CPU
optimizations

Note: A hot-swap chassis allows you to remove a hard drive without having to
power-down your system.

4. RAID Level 0 [Striping]

* RAID-0 is also called Striping. In this level two or more Hard Disks are
combined to appear as one large one to the OS

Example :
=======
I have two HDD of 8GB and 20GB. In RAID-0, both are combined and you get a
combined disk space of 28GB. There is no Data Redudancy and Fault Tolerance.
If one of the HDD fails, you lose all your data.

ADVANTAGES
==========

* One advantage of this is speed since a file is spread [strips] across the
two disks and can be read twice as fast

* Can accomodate very large files

* Can accomodate disks of unequal sizes. When RAID runs out of space on the
smaller [8GB] disk, it then continues the striping using the available space
on the remaining drives. When this occurs, the data access speed is lower for
this portion of data, because the total number of RAID drives available is
reduced. For this reason, RAID 0 is best used with drives of equal size.

DISADVANTAGES
=============

* There is no Data Redundancy and Fault Tolerance.
If one of the HDD fails, you lose all your data.

* You could, however, use one HDD too, if you are stupid enough to do that

+---------+ +---------+
| Block 1 | | Block 1 |
+---------+ +---------+
+---------+ Physical +---------+
| Block 2 | Disk 1 | Block 3 |
+---------+ +---------+
+---------+ +---------+
| Block 3 | RAID-0 | Block 5 |
+---------+ +---------+
==========>
+---------+ +---------+
| Block 4 | | Block 2 |
+---------+ Physical +---------+
Disk 2
+---------+ +---------+
| Block 5 | | Block 4 |
+---------+ +---------+

+---------+ +---------+
| Block 6 | | Block 6 |
+---------+ +---------+

5. RAID Level 1 [Mirroring]

* RAID 1, or "mirroring," has been used longer than any other form of RAID.

* Level 1 provides redundancy by writing identical data to each member disk of
the array, leaving a "mirrored" copy on each disk.

* Here you use two hard disks such that both of them contain exactly the same
information.

* In case of failure of one disk, the server will boot through the second disk.
When the failed disk is replaced, the data is automatically cloned to the new
disk from the surviving disk.

* Level 1 operates with two or more disks that may use parallel access for high
data-transfer rates when reading but more commonly operate independently to
provide high I/O transaction rates.

* RAID 1 also offers the possibility of using a hot standby spare disk that will
be automatically cloned in the event of a disk failure on any of the primary
RAID devices.

* Mirroring remains popular due to its simplicity and high level of data
availability.

ADVANTAGES
==========

* Offers redundancy and more Fault Tolerance.

* Provides very good data reliability and improves performance for
read-intensive applications

DISADVANTAGES
=============

* Total RAID size in GB is equal to that of the smallest disk in the RAID set.
Unlike RAID 0, the extra space on the larger device isn't used.

* RAID 1 offers data redundancy, without the speed advantages of RAID 0.
The server has to send data twice to be written to each of the mirrored disks.
This can saturate data busses and CPU use.

With a hardware-based solution, the server CPU sends the data to the RAID
disk controller once, and the disk controller then duplicates the data to
the mirrored disks.

This makes RAID-capable disk controllers the preferred solution when
implementing RAID 1.

* The storage capacity of the level 1 array is equal to the capacity of one
of the mirrored hard disks in a Hardware RAID or one of the mirrored
partitions in a Software RAID.

+---------+ +---------+ +---------+
| Block 1 | | Block 1 | | Block 1 |
+---------+ RAID-1 +---------+ +---------+
| Block 2 | | Block 2 | | Block 2 |
+---------+ +---------+ +---------+
| Block 3 | | Block 3 | | Block 3 |
+---------+ ========> +---------+ ========> +---------+
| Block 4 | | Block 4 | | Block 4 |
+---------+ +---------+ +---------+
| Block 5 | | Block 5 | | Block 5 |
+---------+ +---------+ +---------+
| Block 6 | | Block 6 | | Block 6 |
+---------+ +---------+ +---------+

Physical Disk 1 Physical Disk 2

6. RAID Level 4 = 0 + 1 w/o parity

* Linux RAID 4 requires a minimum of three disks or partitions and can
survive the loss of one disk only

* RAID 4 combines the high speed provided of RAID 0 with the redundancy of
RAID 1.

* Level 4 uses parity concentrated on a single disk drive to protect data.
RAID 4 operates likes RAID 0 but inserts a special error-correcting or parity
chunk on an additional disk dedicated to this purpose.

* Its major disadvantage is that the data is striped, but the parity info is
not. In other words, any data written to any section of the data portion
of the RAID set must be followed by an update of the parity disk. The parity
disk can therefore act as a bottleneck. For this reason, RAID 4 isn't used
very frequently.
Because the dedicated parity disk represents an inherent bottleneck,
level 4 is seldom used without accompanying technologies such as
write-back caching.

* RAID 4 requires at least three disks in the RAID set and can survive the loss
of a single drive only. When this occurs, the data in it can be recreated on
the fly with the aid of the information on the RAID set's parity disk. When
the failed disk is replaced, it is re-populated with the lost data with the
help of the parity disk's information.

* It is better suited to transaction I/O rather than large file transfers.

* Although RAID level 4 is an option in some RAID partitioning schemes, it is
not an option allowed in Red Hat Linux RAID installations.

* The storage capacity of Hardware RAID level 4 is equal to the capacity of
member disks, minus the capacity of one member disk.

* The storage capacity of Software RAID level 4 is equal to the capacity of
the member partitions, minus the size of one of the partitions if they
are of equal size.

* RAID 4 is not supported by Fedora Linux.

7. RAID 5 = 4 with parity

* Linux RAID 5 requires a minimum of three disks or partitions and can
survive the loss of one disk only

* This is the most common type of RAID. By distributing parity across some
or all of an array's member disk drives, RAID level 5 eliminates the
write bottleneck inherent in level 4.

* The only performance bottleneck is the parity calculation process. With
modern CPUs and Software RAID, that usually is not a very big problem.

* As with level 4, the result is asymmetrical performance, with reads
substantially outperforming writes.

* Level 5 is often used with write-back caching to reduce the asymmetry.

* The storage capacity of Hardware RAID level 5 is equal to the capacity of
member disks, minus the capacity of one member disk.

* The storage capacity of Software RAID level 5 is equal to the capacity
of the member partitions, minus the size of one of the partitions if they
are of equal size.

* RAID 5 improves on RAID 4 by striping the parity data between all the
disks in the RAID set. This avoids the parity disk bottleneck, whilst
maintaining many of the speed features of RAID 0 and the redundancy of
RAID 1. Like RAID 4, RAID 5 can survive the loss of a single disk only.

* RAID 5 is supported by Fedora Linux. Figure below illustrates the data
allocation process in RAID 5.

+---------+
| Block 1 |
+---------+
+---------+ +----------+ +---------+ +-----------------+
| Block 2 | | Block 1 | | Block 2 | |ErrChk Blocks 1+2|
+---------+ +---------+ +---------+ +-----------------+
+---------+ +----------+-----------+ +---------+ +---------+
| Block 3 | RAID-5 | ErrChk Blocks 3+4 | | Block 1 | | Block 3 |
+---------+ +-Block 3+4----------+ +---------+ +---------+
+---------+ +---------+ +-------------------+ +---------+
| Block 4 | | Block 6 | | ErrChk Blocks 6+5 | | Block 5 |
+---------+ +---------+ +-------------------+ +---------+
+---------+ +---------+ +---------+ +------------------+
| Block 5 | | Block 7 | | Block 8 | |ErrChk Blocks 7+8 |
+---------+ +---------+ +---------+ +------------------+
+---------+
| Block 6 |
+---------+

Linear RAID

Linear RAID is a simple grouping of drives to create a larger virtual drive.
In linear RAID, the chunks are allocated sequentially from one member drive,
going to the next drive only when the first is completely filled.

This grouping provides no performance benefit, as it is unlikely
that any I/O operations will be split between member drives. Linear RAID also
offers no redundancy and, in fact, decreases reliability if any one member
drive fails, the entire array cannot be used. The capacity is the total of all
member disks.

Note : RAID level 1 comes at a high cost because you write the same info to
all of the disks in the array, which wastes drive space.
For example, if you have RAID level 1 set up so that your
root (/) partition exists on two 40G drives, you have 80G total but
are only able to access 40G of that 80G. The other 40G acts like a
mirror of the first 40G.

Note : Parity information is calculated based on the contents of the rest of
the member disks in the array. This information can then be used to
reconstruct data when one disk in the array fails. The reconstructed
data can then be used to satisfy I/O requests to the failed disk before
it is replaced and to repopulate the failed disk after it has been
replaced.

Note : RAID level 4 takes up the same amount of space as RAID level 5, but
level 5 has more advantages. For this reason, level 4 is not supported.

Advanced Squid

2008-03-29T21:54:00.000+05:30

Squid is a free caching proxy server that runs on Linux and many other operating systems. Many Linux users who have used Squid have taken advantage of its simple setup, and ignore or overlook its advanced features. Here's an introduction to some of those features and how to use them.
I'll assume that you have already set up a basic Squid system. If you need help on this, see the "Transparent Proxy with Linux and Squid mini-HOWTO" and the Squid quick start guide. Once you know the basics, you can move on to advanced topics such as access control lists (ACL), proxy authentication schemes, delay pools, and blocking pornography.

Squid has one primary configuration file, squid.conf. This file is generally located in /etc/squid/, or if you compiled Squid from source, the default location is /usr/local/squid/etc/. You'll be editing this file, so it's wise to make a backup copy of it before you make any changes.

Access control lists

The primary use of ACLs is to control access, but they can also be used to route requests through a hierarchy, control request rewriting, and manage quality of service.

Access controls are divided into two parts: elements and rules. ACL elements are things such as IP addresses, port numbers, hostnames, and URL patterns. Each ACL element has a name, which you refer to when writing the access list rules. The basic syntax of an ACL element is:

ACLname type value1 value2

Squid has more than 20 ACL types, including types for source and destination IP addresses, time, URLs, port numbers, and transfer protocols. See the Squid Configuration Manual for a full list of types.

After defining the ACL elements, the next step is to combine them with Access list rules. Rules combine elements to allow or deny certain actions. The syntax for an access control rule is:

access_list allow|deny [!]ACLname
For example, the rule:
http_access allow MyClients

tells Squid to allow access from the host or hosts defined under the name MyClients. The optional exclamation point is a standard negation operator, used to reverse the logic of the ACL. If this seems confusing, the following examples should help.

Restricting access to local network users

You should always limit access to your proxy server to local IP addresses, unless you have a specific need to allow external users. This can save you large bandwidth bills, from outsiders using your machine as a proxy. A simple way to do this is to write an ACL that contains your IP address space and then allow HTTP requests for that ACL and deny all others:

acl All src 0/0
acl PrivateNet src 192.168.0.0/24 192.168.1.0/24
http_access allow PrivateNet
http_access deny All

Squid makes one pass through the configuration file, reading the ACLs and rules in order. This means that you must define an ACL before you make a rule applying it, and the order of the http_access rules is important. Incoming requests are checked in the order in which the rules are written. If the first rule allows the request, the remaining requests are not read. If the first rule blocks the request, Squid passes on to the next one, and so on. Your last http_access line should always be a deny All, so that a request which is not permitted by any of the previous rules is blocked by default. If you change this to allow All, all your rules become meaningless, since Squid will allow the request at the end. The default squid.conf configuration file contains some important access controls. Try not to change these before you understand what they do. When you edit squid.conf for the first time, look for this comment:

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

Insert your new rules below this comment, and before the http_access deny All line.

Blocking specific computers

It is often necessary to block a particular IP address. At our university, for example, if a student uses excessive bandwidth, we block his computer for a few days. Until you can solve the problem at the source, you can block requests coming to Squid with this configuration:

acl All src 0/0
acl PrivateNet src 192.168.0.0/24 192.168.1.0/24
acl ProblemHost src 192.168.0.15
http_access deny ProblemHost
http_access allow PrivateNet
http_access deny All

This will block requests from the IP address 192.168.0.15. You can also block an IP range, such as 192.168.0.0/24.

Restricting usage to specified Web sites during working hours

You can set up a simple ACL to restrict Internet usage to work-related sites during working hours. To do this, you need to make a list of allowed sites and save it as a file with the domain names on each line. For example:

#Allowed Sites
www.cnn.com
www.news.google.com
www.bbc.co.uk
www.newsforge.com
and other allowed sites...

Once you have your allow list ready, use the following ACL to restrict usage:

acl All src 0/0
acl PrivateNet src 192.168.0.0/24 192.168.1.0/24
acl AllowedSites dstdomain "/usr/local/squid/etc/allowed-sites"
acl WorkingHours time D 08:00-17:30
http_access allow WorkingHours AllowedSites
http_access deny All
Blocking pornography

Pornographic sites are quite a headache for many organizations. While many specialized free and commercial packages exist for filtering content, you can use Squid to block pornography as well.

The hardest part about using Squid to deny access to pornography is coming up with the list of sites that should be blocked. If you want a ready-made list, the Access Controls section of the Squid FAQ has links to freely available lists.

The ACL you have to write for such a list depends on the content of the list. If the list contains regular expressions, you'll need to use the following ACL:

acl PornSites url_regex "/usr/local/squid/etc/pornlist"
http_access deny PornSites

If the list contains hostnames, the url_regex will have to be changed to dstdomain, which tells Squid to match the entire hostname instead of the words in the hostname:

acl PornSites dstdomain "/usr/local/squid/etc/pornlist"
http_access deny PornSites

These methods are fine for casual use. If you are really serious about blocking such sites, you might want to look at specialized software, such as SquidGuard or Dansguardian.

Proxy authentication

Proxy authentication is a complex subject, due to the various types of proxy authentication schemes available. I describe a simple user authentication scheme below, but there are many more schemes available, and the best one will vary according to your specific needs.

Squid currently supports three techniques for receiving user credentials: HTTP Basic and Digest and NTLM. Basic authentication has been around for a long time. Though this is what I use in this example, you should know that it is a very insecure protocol, since the usernames and passwords are sent over the network in clear text. Anyone who runs a packet analyzer on your network can get the passwords. Still, it's a good place to start, and for smaller networks, where security is not a major problem, it works well.

To use proxy authentication, Squid needs to be configured to spawn a number of external helper processes. The Squid source code includes some programs that authenticate against a number of standard databases. The auth_param directive controls the configuration of all helper programs.

The order of the auth_param directive and proxy_auth ACL is extremely important. Remember that Squid reads the config file in one pass, and in order. If you don't put the proxy authentication ACLs in the proper order, you could end up allowing (or denying) all access. To use proxy authentication, you must define at least one authentication helper before any proxy_auth ACLs. If you don't, Squid will print an error message to the logs and start up anyway, and all user requests may be denied. If you try to set up proxy authentication and find that it's not working, look at the logs to make sure that the problem does not lie in the order of the ACLs.

HTTP Basic authentication supports the following auth_param parameters:

l auth_param basic program command
l auth_param basic children number
l auth_param basic realm string
l auth_param basic credentialsttl time-specification

The program parameter specifies the command, including arguments, for the helper program. This is generally the pathname to one of the authentication helper programs. By default, the path is /usr/local/squid/libexec.

The children parameter tells Squid how many helper processes to use. The default value is 5, which is a good starting point if you don't know how many helpers Squid needs to handle the load. For a 400-user network, I use a value of 25. You should check your cache.log to make sure that there are no warning messages about too few helper processes, and increase the number of helper processes if there are warnings.

The realm parameter is the authentication realm string that the proxy server should present to the user when prompting for a username and password. Use something simple, such as "Orgname Proxy Server."

The credentialsttl parameter specifies the amount of time that Squid internally caches authentication results. A larger "time to live" value reduces the load on the external authenticator processes, but increases the amount of time until Squid detects changes to the authentication database. If you have a relatively fixed user base, set this high, but if the user base is transient, as in a public library, use a lower value. The default TTL value is two hours.

A complete setup would look like this:

auth_param basic program /usr/local/squid/libexec/ncsa_auth /usr/local/squid/etc/passwd
auth_param basic children 10
auth_param basic realm NLU Proxy Server
auth_param basic credentialsttl 3 hour
acl Students proxy_auth REQUIRED
http_access allow Students

For this example I have used the NCSA authentication helper, which is a simple authentication method that stores usernames and passwords in a single text file, similar to the /etc/passwd file. You pass the path to the password file as the program's single command-line argument in Squid.conf:

auth_param basic program /usr/local/squid/libexec/ncsa_auth /usr/local/squid/etc/passwd

To create and update the file, you can use the htpasswd program. If you have the Apache Web server installed, htpasswd should also be installed; if not, download it from the Squid Web site. To create a file, the command is htpasswd -c passwdfile user.

To add users and change their passwords, the command is htpasswd passwdfile username.

htpasswd will prompt you for a password. If you want to allow users to change their own passwords, you can use the chpasswd CGI script, which is also available on the Squid Web site.

There are several other authentication helpers you can use with Basic authentication. For example, you can authenticate against a LDAP server, Windows Domain, or Samba domain.

Conclusion

If you want to learn more about Squid, I recommend a book called Squid: The Definitive Guide , written by Duane Wessels and published by O'Reilly and Associates. Squid is a versatile and robust proxy server, and it can be used in very complex configurations. I hope that this introduction will help you in using some of Squid's more advanced features.

Postfix

2008-03-29T21:36:00.000+05:30

Postfix Basic Configuration

Introduction
Postfix has several hundred configuration parameters that are controlled via the main.cf file. Fortunately, all parameters have sensible default values. In many cases, you need to configure only two or three parameters before you can start to play with the mail system

Postfix configuration files

Postfix configuration files
By default, Postfix configuration files are in /etc/postfix. The two most important files are main.cf and master.cf; these files must be owned by root. Giving someone else write permission to main.cf or master.cf (or to their parent directories) means giving root privileges to that person.

In /etc/postfix/main.cf you will have to set up a minimal number of configuration parameters. Postfix configuration parameters resemble shell variables, with two important differences: the first one is that Postfix does not know about quotes like the UNIX shell does.

You specify a configuration parameter as:

/etc/postfix/main.cf:
parameter = value
and you use it by putting a "$" character in front of its name:

/etc/postfix/main.cf:
other_parameter = $parameter
You can use $parameter before it is given a value (that is the second main difference with UNIX shell variables). The Postfix configuration language uses lazy evaluation, and does not look at a parameter value until it is needed at runtime.

Postfix uses database files for access control, address rewriting and other purposes. The DATABASE_README file gives an introduction to how Postfix works with Berkeley DB, LDAP or SQL and other types. Here is a common example of how Postfix invokes a database:

/etc/postfix/main.cf:
virtual_alias_maps = hash:/etc/postfix/virtual
Whenever you make a change to the main.cf or master.cf file, execute the following command as root in order to refresh a running mail system:

# postfix reload

What domain name to use in outbound mail

The myorigin parameter specifies the domain that appears in mail that is posted on this machine. The default is to use the local machine name, $myhostname, which defaults to the name of the machine. Unless you are running a really small site, you probably want to change that into $mydomain, which defaults to the parent domain of the machine name.

For the sake of consistency between sender and recipient addresses, myorigin also specifies the domain name that is appended to an unqualified recipient address.

Examples (specify only one of the following):

/etc/postfix/main.cf:
myorigin = $myhostname (default: send mail as "user@$myhostname")
myorigin = $mydomain (probably desirable: "user@$mydomain")
What domains to receive mail for
The mydestination parameter specifies what domains this machine will deliver locally, instead of forwarding to another machine. The default is to receive mail for the machine itself. See the VIRTUAL_README file for how to configure Postfix for hosted domains.

You can specify zero or more domain names, "/file/name" patterns and/or "type:table" lookup tables (such as hash:, btree:, nis:, ldap:, or mysql:), separated by whitespace and/or commas. A "/file/name" pattern is replaced by its contents; "type:table" requests that a table lookup is done and merely tests for existence: the lookup result is ignored.

IMPORTANT: If your machine is a mail server for its entire domain, you must list $mydomain as well.

Example 1: default setting.

/etc/postfix/main.cf:
mydestination = $myhostname localhost.$mydomain localhost
Example 2: domain-wide mail server.

/etc/postfix/main.cf:
mydestination = $myhostname localhost.$mydomain localhost $mydomain
Example 3: host with multiple DNS A records.

/etc/postfix/main.cf:
mydestination = $myhostname localhost.$mydomain localhost
www.$mydomain ftp.$mydomain
Caution: in order to avoid mail delivery loops, you must list all hostnames of the machine, including $myhostname, and localhost.$mydomain.

What clients to relay mail from

By default, Postfix will forward mail from clients in authorized network blocks to any destination. Authorized networks are defined with the mynetworks configuration parameter. The default is to authorize all clients in the IP subnetworks that the local machine is attached to.

IMPORTANT: If your machine is connected to a wide area network then your default mynetworks setting may be too friendly.

Examples (specify only one of the following):

/etc/postfix/main.cf:
mynetworks_style = subnet (default: authorize subnetworks)
mynetworks_style = host (safe: authorize local machine only)
mynetworks = 127.0.0.0/8 (safe: authorize local machine only)
mynetworks = 127.0.0.0/8 168.100.189.2/32 (authorize local machine)
You can specify the trusted networks in the main.cf file, or you can let Postfix do the work for you. The default is to let Postfix do the work. The result depends on the mynetworks_style parameter value.

Specify "mynetworks_style = host" when Postfix should forward mail from only the local machine.

Specify "mynetworks_style = subnet" (the default) when Postfix should forward mail from SMTP clients in the same IP subnetworks as the local machine. On Linux, this works correctly only with interfaces specified with the "ifconfig" command.

Specify "mynetworks_style = class" when Postfix should forward mail from SMTP clients in the same IP class A/B/C networks as the local machine. Don't do this with a dialup site - it would cause Postfix to "trust" your entire provider's network. Instead, specify an explicit mynetworks list by hand, as described below.

Alternatively, you can specify the mynetworks list by hand, in which case Postfix ignores the mynetworks_style setting. To specify the list of trusted networks by hand, specify network blocks in CIDR (network/mask) notation, for example:

/etc/postfix/main.cf:
mynetworks = 168.100.189.0/28, 127.0.0.0/8
You can also specify the absolute pathname of a pattern file instead of listing the patterns in the main.cf file.

What destinations to relay mail to

By default, Postfix will forward mail from strangers (clients outside authorized networks) to authorized remote destinations only. Authorized remote destinations are defined with the relay_domains configuration parameter. The default is to authorize all domains (and subdomains) of the domains listed with the mydestination parameter.

Examples (specify only one of the following):

/etc/postfix/main.cf:
relay_domains = $mydestination (default)
relay_domains = (safe: never forward mail from strangers)
relay_domains = $mydomain (forward mail to my domain and subdomains)

What delivery method: direct or indirect

By default, Postfix tries to deliver mail directly to the Internet. Depending on your local conditions this may not be possible or desirable. For example, your system may be turned off outside office hours, it may be behind a firewall, or it may be connected via a provider who does not allow direct mail to the Internet. In those cases you need to configure Postfix to deliver mail indirectly via a relay host.

Examples (specify only one of the following):

/etc/postfix/main.cf:
relayhost = (default: direct delivery to Internet)
relayhost = $mydomain (deliver via local mailhub)
relayhost = [mail.$mydomain] (deliver via local mailhub)
relayhost = [mail.isp.tld] (deliver via provider mailhub)
The form enclosed with [] eliminates DNS MX lookups. Don't worry if you don't know what that means. Just be sure to specify the [] around the mailhub hostname that your ISP gave to you, otherwise mail may be mis-delivered.

The STANDARD_CONFIGURATION_README file has more hints and tips for firewalled and/or dial-up networks.

What trouble to report to the postmaster

You should set up a postmaster alias in the aliases(5) table that directs mail to a human person. The postmaster address is required to exist, so that people can report mail delivery problems. While you're updating the aliases(5) table, be sure to direct mail for the super-user to a human person too.

/etc/aliases:
postmaster: you
root: you
Execute the command "newaliases" after changing the aliases file. Instead of /etc/aliases, your alias file may be located elsewhere. Use the command "postconf alias_maps" to find out.

The Postfix system reports problems to the postmaster alias. You may not be interested in all types of trouble reports, so this reporting mechanism is configurable. The default is to report only serious problems (resource, software) to postmaster:

Default setting:

/etc/postfix/main.cf:
notify_classes = resource, software
The meaning of the classes is as follows:

bounce
Inform the postmaster of undeliverable mail. Either send the postmaster a copy of undeliverable mail that is returned to the sender, or send a transcript of the SMTP session when Postfix rejected mail. For privacy reasons, the postmaster copy of undeliverable mail is truncated after the original message headers. This implies "2bounce" (see below). See also the luser_relay feature. The notification is sent to the address specified with the bounce_notice_recipient configuration parameter (default: postmaster).
2bounce
When Postfix is unable to return undeliverable mail to the sender, send it to the postmaster instead (without truncating the message after the primary headers). The notification is sent to the address specified with the 2bounce_notice_recipient configuration parameter (default: postmaster).
delay
Inform the postmaster of delayed mail. In this case, the postmaster receives message headers only. The notification is sent to the address specified with the delay_notice_recipient configuration parameter (default: postmaster).
policy
Inform the postmaster of client requests that were rejected because of (UCE) policy restrictions. The postmaster receives a transcript of the SMTP session. The notification is sent to the address specified with the error_notice_recipient configuration parameter (default: postmaster).
protocol
Inform the postmaster of protocol errors (client or server side) or attempts by a client to execute unimplemented commands. The postmaster receives a transcript of the SMTP session. The notification is sent to the address specified with the error_notice_recipient configuration parameter (default: postmaster).
resource
Inform the postmaster of mail not delivered due to resource problems (for example, queue file write errors). The notification is sent to the address specified with the error_notice_recipient configuration parameter (default: postmaster).
software
Inform the postmaster of mail not delivered due to software problems. The notification is sent to the address specified with the error_notice_recipient configuration parameter (default: postmaster).

Proxy/NAT external network addresses

Some mail servers are connected to the Internet via a network address translator (NAT) or proxy. This means that systems on the Internet connect to the address of the NAT or proxy, instead of connecting to the network address of the mail server. The NAT or proxy forwards the connection to the network address of the mail server, but Postfix does not know this.

If you run a Postfix server behind a proxy or NAT, you need to configure the proxy_interfaces parameter and specify all the external proxy or NAT addresses that Postfix receives mail on. You may specify symbolic hostnames instead of network addresses.

IMPORTANT: You must specify your proxy/NAT external addresses when your system is a backup MX host for other domains, otherwise mail delivery loops will happen when the primary MX host is down.

Example: host behind NAT box running a backup MX host.

/etc/postfix/main.cf:
proxy_interfaces = 1.2.3.4 (the proxy/NAT external network address)

What you need to know about Postfix logging

Postfix daemon processes run in the background, and log problems and normal activity to the syslog daemon. The syslogd process sorts events by class and severity, and appends them to logfiles. The logging classes, levels and logfile names are usually specified in /etc/syslog.conf. At the very least you need something like:

/etc/syslog.conf:
mail.err /dev/console
mail.debug /var/log/maillog
After changing the syslog.conf file, send a "HUP" signal to the syslogd process.

IMPORTANT: many syslogd implementations will not create files. You must create files before (re)starting syslogd.

IMPORTANT: on Linux you need to put a "-" character before the pathname, e.g., -/var/log/maillog, otherwise the syslogd process will use more system resources than Postfix.

Hopefully, the number of problems will be small, but it is a good idea to run every night before the syslog files are rotated:

# postfix check
# egrep '(reject|warning|error|fatal|panic):' /some/log/file
The first line (postfix check) causes Postfix to report file permission/ownership discrepancies.

The second line looks for problem reports from the mail software, and reports how effective the relay and junk mail access blocks are. This may produce a lot of output. You will want to apply some postprocessing to eliminate uninteresting information.

The DEBUG_README document describes the meaning of the "warning" etc. labels in Postfix logging.

Running Postfix daemon processes chrooted

Postfix daemon processes can be configured (via the master.cf file) to run in a chroot jail. The processes run at a fixed low privilege and with file system access limited to the Postfix queue directories (/var/spool/postfix). This provides a significant barrier against intrusion. The barrier is not impenetrable (chroot limits file system access only), but every little bit helps.

With the exception of Postfix daemons that deliver mail locally and/or that execute non-Postfix commands, every Postfix daemon can run chrooted.

Sites with high security requirements should consider to chroot all daemons that talk to the network: the smtp(8) and smtpd(8) processes, and perhaps also the lmtp(8) client. The author's own porcupine.org mail server runs all daemons chrooted that can be chrooted.

The default /etc/postfix/master.cf file specifies that no Postfix daemon runs chrooted. In order to enable chroot operation, edit the file /etc/postfix/master.cf, and follow instructions in the file. When you're finished, execute "postfix reload" to make the change effective.

Note that a chrooted daemon resolves all filenames relative to the Postfix queue directory (/var/spool/postfix). For successful use of a chroot jail, most UNIX systems require you to bring in some files or device nodes. The examples/chroot-setup directory in the source code distribution has a collection of scripts that help you set up Postfix chroot environments on different operating systems.

Additionally, you almost certainly need to configure syslogd so that it listens on a socket inside the Postfix queue directory. Examples of syslogd command line options that achieve this for specific systems:

FreeBSD: syslogd -l /var/spool/postfix/var/run/log

Linux, OpenBSD: syslogd -a /var/spool/postfix/dev/log

If you run Postfix on a virtual network interface, or if your machine runs other mailers on virtual interfaces, you'll have to look at the other parameters listed here as well:

My own hostname

The myhostname parameter specifies the fully-qualified domain name of the machine running the Postfix system. $myhostname appears as the default value in many other Postfix configuration parameters.

By default, myhostname is set to the local machine name. If your local machine name is not in fully-qualified domain name form, or if you run Postfix on a virtual interface, you will have to specify the fully-qualified domain name that the mail system should use.

Alternatively, if you specify mydomain in main.cf, then Postfix will use its value to generate a fully-qualified default value for the myhostname parameter.

Examples (specify only one of the following):

/etc/postfix/main.cf:
myhostname = host.local.domain (machine name is not FQDN)
myhostname = host.virtual.domain (virtual interface)
myhostname = virtual.domain (virtual interface)

My own domain name

The mydomain parameter specifies the parent domain of $myhostname. By default, it is derived from $myhostname by stripping off the first part (unless the result would be a top-level domain).

Conversely, if you specify mydomain in main.cf, then Postfix will use its value to generate a fully-qualified default value for the myhostname parameter.

Examples (specify only one of the following):

/etc/postfix/main.cf:
mydomain = local.domain
mydomain = virtual.domain (virtual interface)

My own network addresses

The inet_interfaces parameter specifies all network interface addresses that the Postfix system should listen on; mail addressed to "user@[network address]" will be delivered locally, as if it is addressed to a domain listed in $mydestination.

You can override the inet_interfaces setting in the Postfix master.cf file by prepending an IP address to a server name.

The default is to listen on all active interfaces. If you run mailers on virtual interfaces, you will have to specify what interfaces to listen on.

IMPORTANT: If you run MTAs on virtual interfaces you must specify explicit inet_interfaces values for the MTA that receives mail for the machine itself: this MTA should never listen on the virtual interfaces or you would have a mailer loop when a virtual MTA is down.

Example: default setting.

/etc/postfix/main.cf:
inet_interfaces = all
Example: host running one or more virtual mailers. For each Postfix instance, specify only one of the following.

/etc/postfix/main.cf:
inet_interfaces = virtual.host.tld (virtual Postfix)
inet_interfaces = $myhostname localhost... (non-virtual Postfix)
Note: you need to stop and start Postfix after changing this parameter.

Postfix SMTP relay and access control

Introduction
The Postfix SMTP server receives mail from the network and is exposed to the big bad world of junk email and viruses. This document introduces the built-in and external methods that control what SMTP mail Postfix will accept, what mistakes to avoid, and how to test your configuration

Relay control, junk mail control, and per-user policies
In a distant past, the Internet was a friendly environment. Mail servers happily forwarded mail on behalf of anyone towards any destination. On today's Internet, spammers abuse servers that forward mail from arbitrary systems, and abused systems end up on anti-spammer blacklists. See, for example, the information on http://www.mail-abuse.org/ and other websites.

By default, Postfix has a moderately restrictive approach to mail relaying. Postfix forwards mail only from clients in trusted networks, or to domains that are configured as authorized relay destinations. For a description of the default policy, see the smtpd_recipient_restrictions parameter in the postconf(5) manual page, and the information that is referenced from there.

Most of the Postfix SMTP server access controls are targeted at stopping junk email.

Protocol oriented: some SMTP server access controls block mail by being very strict with respect to the SMTP protocol; these catch poorly implemented and/or poorly configured junk email software, as well as email worms that come with their own non-standard SMTP client implementations. Protocol-oriented access controls become less useful over time as spammers and worm writers learn to read RFC documents.

Blacklist oriented: some SMTP server access controls query blacklists with known to be bad sites such as open mail relays, open web proxies, and home computers that have been compromised and that are under remote control by criminals. The effectiveness of these blacklists depends on how complete and how up to date they are.

Threshold oriented: some SMTP server access controls attempt to raise the bar by either making the client do more work (greylisting) or by asking for a second opinion (SPF and sender/recipient address verification). The greylisting and SPF policies are implemented externally, and are the subject of the SMTPD_POLICY_README document. Sender/recipient address verification is the subject of the ADDRESS_VERIFICATION_README document.

Unfortunately, all junk mail controls have the possibility of falsely rejecting legitimate mail. This can be a problem for sites with many different types of users. For some users it is unacceptable when any junk email slips through, while for other users the world comes to an end when a single legitimate email message is blocked. Because there is no single policy that is "right" for all users, Postfix supports different SMTP access restrictions for different users. This is described in the RESTRICTION_CLASS_README document.

Restrictions that apply to all SMTP mail
Besides the restrictions that can be made configurable per client or per user as described in the next section, Postfix implements a few restrictions that apply to all SMTP mail.

The built-in header_checks and body_checks content restrictions, as described in the BUILTIN_FILTER_README document. This happens while Postfix receives mail, before it is stored in the incoming queue.

The external before-queue content restrictions, as described in the SMTPD_PROXY_README document. This happens while Postfix receives mail, before it is stored in the incoming queue.

Requiring that the client sends the HELO or EHLO command before sending the MAIL FROM or ETRN command. This may cause problems with home-grown applications that send mail. For this reason, the requirement is disabled by default ("smtpd_helo_required = no").

Disallowing illegal syntax in MAIL FROM or RCPT TO commands. This may cause problems with home-grown applications that send mail, and with ancient PC mail clients. For this reason, the requirement is disabled by default ("strict_rfc821_envelopes = no").

Disallowing RFC 822 address syntax (example: "MAIL FROM: the dude ").

Disallowing addresses that are not enclosed with <> (example: "MAIL FROM: dude@example.com").

Rejecting mail from a non-existent sender address. This form of egress filtering helps to slow down worms and other malware, but may cause problems with home-grown software that sends out mail software with an unreplyable address. For this reason the requirement is disabled by default ("smtpd_reject_unlisted_sender = no").

Rejecting mail for a non-existent recipient address. This form of ingress filtering helps to keep the mail queue free of undeliverable MAILER-DAEMON messages. This requirement is enabled by default ("smtpd_reject_unlisted_recipient = yes").

Getting selective with SMTP access restriction lists
Postfix allows you to specify lists of access restrictions for each stage of the SMTP conversation. Individual restrictions are described in the postconf(5) manual page.

Examples of simple restriction lists are:

/etc/postfix/main.cf:
# Allow connections from trusted networks only.
smtpd_client_restrictions = permit_mynetworks, reject

# Don't talk to mail systems that don't know their own hostname.
# With Postfix < 2.3, specify reject_unknown_hostname.
smtpd_helo_restrictions = reject_unknown_helo_hostname

# Don't accept mail from domains that don't exist.
smtpd_sender_restrictions = reject_unknown_sender_domain

# Whitelisting: local clients may specify any destination. Others may not.
smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination

# Block clients that speak too early.
smtpd_data_restrictions = reject_unauth_pipelining

# Enforce mail volume quota via policy service callouts.
smtpd_end_of_data_restrictions = check_policy_service unix:private/policy
Each restriction list is evaluated from left to right until some restriction produces a result of PERMIT, REJECT or DEFER (try again later). The end of the list is equivalent to a PERMIT result. By placing a PERMIT restriction before a REJECT restriction you can make exceptions for specific clients or users. This is called whitelisting; the last example above allows mail from local networks but otherwise rejects mail to arbitrary destinations.

The table below summarizes the purpose of each SMTP access restriction list. All lists use the exact same syntax; they differ only in the time of evaluation and in the effect of a REJECT or DEFER result.

Restriction list name Status Effect of REJECT or DEFER result
smtpd_client_restrictions Optional Reject all client commands
smtpd_helo_restrictions Optional Reject HELO/EHLO information
smtpd_sender_restrictions Optional Reject MAIL FROM information
smtpd_recipient_restrictions Required Reject RCPT TO information
smtpd_data_restrictions Optional Reject DATA command
smtpd_end_of_data_restrictions Optional Reject END-OF-DATA command
smtpd_etrn_restrictions Optional Reject ETRN command

Delayed evaluation of SMTP access restriction lists
Early Postfix versions evaluated SMTP access restrictions lists as early as possible. The client restriction list was evaluated before Postfix sent the "220 $myhostname..." greeting banner to the SMTP client, the helo restriction list was evaluated before Postfix replied to the HELO (EHLO) command, the sender restriction list was evaluated before Postfix replied to the MAIL FROM command, and so on. This approach turned out to be difficult to use.

Current Postfix versions postpone the evaluation of client, helo and sender restriction lists until the RCPT TO or ETRN command. This behavior is controlled by the smtpd_delay_reject parameter. Restriction lists are still evaluated in the proper order of (client, helo, etrn) or (client, helo, sender, recipient, data, or end-of-data) restrictions. When a restriction list (example: client) evaluates to REJECT or DEFER the other restriction lists (example: helo, sender, etc.) are skipped.

Around the time that smtpd_delay_reject was introduced, Postfix was also changed to support mixed restriction lists that combine information about the client, helo, sender and recipient or etrn command.

Benefits of delayed restriction evaluation, and of restriction mixing:

Some SMTP clients do not expect a negative reply early in the SMTP session. When the bad news is postponed until the RCPT TO reply, the client goes away as it is supposed to, instead of hanging around until a timeout happens, or worse, going into an endless connect-reject-connect loop.

Postfix can log more useful information. For example, when Postfix rejects a client name or address and delays the action until the RCPT TO command, it can log the sender and the recipient address. This is more useful than logging only the client hostname and IP address and not knowing whose mail was being blocked.

Mixing is needed for complex whitelisting policies. For example, in order to reject local sender addresses in mail from non-local clients, you need to be able to mix restrictions on client information with restrictions on sender information in the same restriction list. Without this ability, many per-user access restrictions would be impossible to express.

Dangerous use of smtpd_recipient_restrictions
By now the reader may wonder why we need smtpd client, helo or sender restrictions, when their evaluation is postponed until the RCPT TO or ETRN command. Some people recommend placing ALL the access restrictions in the smtpd_recipient_restrictions list. Unfortunately, this can result in too permissive access. How is this possible?

The purpose of the smtpd_recipient_restrictions feature is to control how Postfix replies to the RCPT TO command. If the restriction list evaluates to REJECT or DEFER, the recipient address is rejected; no surprises here. If the result is PERMIT, then the recipient address is accepted. And this is where surprises can happen.

Here is an example that shows when a PERMIT result can result in too much access permission:

1 /etc/postfix/main.cf:
2 smtpd_recipient_restrictions =
3 permit_mynetworks
4 check_helo_access hash:/etc/postfix/helo_access
5 reject_unknown_helo_hostname
6 reject_unauth_destination
7 /etc/postfix/helo_access:
8 localhost.localdomain PERMIT
Line 5 rejects mail from hosts that don't specify a proper hostname in the HELO command (with Postfix < 2.3, specify reject_unknown_hostname). Lines 4 and 9 make an exception to allow mail from some machine that announces itself with "HELO localhost.localdomain".

The problem with this configuration is that smtpd_recipient_restrictions evaluates to PERMIT for EVERY host that announces itself as "localhost.localdomain", making Postfix an open relay for all such hosts.

In order to avoid surprises like these with smtpd_recipient_restrictions, you should place non-recipient restrictions AFTER the reject_unauth_destination restriction, not before. In the above example, the HELO based restrictions should be placed AFTER reject_unauth_destination, or better, the HELO based restrictions should be placed under smtpd_helo_restrictions where they can do no harm.

SMTP access rule testing
Postfix has several features that aid in SMTP access rule testing:

soft_bounce
This is a safety net that changes SMTP server REJECT actions into DEFER (try again later) actions. This keeps mail queued that would otherwise be returned to the sender. Specify "soft_bounce = yes" in the main.cf file to prevent the Postfix SMTP server from rejecting mail permanently, by changing all 5xx SMTP reply codes into 4xx.

warn_if_reject
This is a different safety net that changes SMTP server REJECT actions into warnings. Instead of rejecting a command, Postfix logs what it would reject. Specify "warn_if_reject" in an SMTP access restriction list, before the restriction that you want to test without actually rejecting mail.

XCLIENT
With this Postfix 2.1 feature, authorized SMTP clients can impersonate other systems, so that you can do realistic SMTP access rule tests. Examples of how to impersonate other systems for access rule testing are given at the end of the XCLIENT_README document

Active/Active with different services on both nodes

2008-03-29T21:31:00.000+05:30

You might rightly think that the active/passive scenario is rather inefficient, because assuming your systems are generally reliable, you have a perfectly good server sitting doing nothing. Therefore, you might prefer to balance the load somewhat using an active/active configuration, where one node "normally" runs some services, and the other node "normally" runs some other (different) services. (See also A Two IP address Active/Active Configuration)

There are a couple of things you need to bear in mind before doing this:

If one node can comfortably cope with the load of running all the services, there isn't necessarily much point in trying to split them. However:
If one node can't cope with the load, then consider what's going to happen when you need to failover all services to one node

(these considerations are general to HA, not DRBD-specific)

However, assuming this is what you want to do, and that the services running on each node each require a shared filesystem (i.e. DRBD), then you need to take the following steps:

Make sure you have separate DRBD resources (drbd0, drbd1...) set up. Since it's not currently possible with DRBD to have a shared filesystem mounted on both primary and secondary nodes (unless you simulate it using cross-mounting with NFS or similar, which is way beyond the scope of this document), we need to separate the filesystem into groups according to which applications are going to be "grouped" together on the nodes when both nodes are up. For maximum flexibility in assigning nodes to applications, you might well want to have a DRBD share per application, assuming of course that no two applications need to access the same share.

If the DRBD devices are on the same disk spindle, use sync groups to ensure that DRBD sync happens at a reasonable speed
Configure Heartbeat appropriately
Let's run through these steps. We'll assume by way of example that you want to run MySQL and Apache. You need one DRBD share for the MySQL data, and one for the Apache data. The nodes will be called 'node1' and 'node2'. We decide to configure it as follows:

share
DRBD resource
DRBD device
Physical device
mountpoint

Apache
r0
/dev/drbd0
/dev/sda1
/ha/web

MySQL
r1
/dev/drbd1
/dev/sda2
/ha/mysql

/dev/drbdX have been /dev/nbX in drbd-0.6.x and older, see also DRBD/QuickStart07

Let's assume, to complicate things, that you have a limited number of physical disks and therefore are forced to have drbd0 and drbd1 on the same spindle (i.e. same hard disk) as shown in the table above (both resources are on /dev/sda in this case). This isn't ideal, but is OK thanks to the 'sync-groups' option. This governs the order in which DRBD resources synchronise (normally, they would sync in parallel, which is going to kill performance if you have got two resources on one disk as the drive would be constantly seeking backwards and forwards reading/writing from/to the two resources). So we use sync-groups to make one resource sync first. It doesn't really matter which goes first. Here's a snippet of a possible drbd.conf. NOTE Only the relevant options are shown here for clarity; you need all your usual options such as disk-size, sync-max etc. in there too:

Syntax was different with drbd-0.6
Just adopting the well commented example drbd.conf to your needs should be easy, though.

# Our web share
resource web {
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; } # or panic, ...
syncer {
group 0;
rate 6M;
}
on node1 {
device /dev/drbd0;
disk /dev/sda1;
address 192.168.99.1:7788;
meta-disk /dev/sdb1[0];
}
on node2 {
device /dev/drbd0;
disk /dev/sda1;
address 192.168.99.2:7788;
meta-disk /dev/sdb1[0];
}
}

# Our MySQL share
resource db {
protocol C;
incon-degr-cmd "echo '!DRBD! pri on incon-degr' | wall ; sleep 60 ; halt -f";
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; } # or panic, ...
syncer {
group 1;
rate 6M;
}
on node1 {
device /dev/drbd1;
disk /dev/sda2;
address 192.168.99.1:7789;
meta-disk /dev/sdb1[1];
}
on node2 {
device /dev/drbd1;
disk /dev/sda2;
address 192.168.99.2:7789;
meta-disk /dev/sdb1[1];
}
}

In the above example, drbd0 will sync first and drbd1 second.

Now, for the heartbeat config. The resources section in haresources is going to look something like this: (you're probably going to need other resources too, like for example IP addresses; this is a simple example)

node1 drbddisk::web Filesystem::/dev/drbd0::/ha/web::ext3 httpd
node2 drbddisk::db Filesystem::/dev/drbd1::/ha/mysql::ext3 mysqld

Note how node1 will "normally" run Apache using the drbd0 resource, and node2 will "normally" run MySQL with the drbd1 resource. If failover occurs, obviously one node will run both, and become DRBD primary for both resources (and have both shares mounted).

Keeping config files in syncAn obvious question you might ask is "how do I keep config files in sync, e.g. for Apache". The simple answer is that you should probably look at using rsync, scp or some other similar method to keep your configs in sync.

If you have many config files (or "cluster files") to keep in sync over arbitrary many hosts, have a look at csync2 by LinBit

Active/Passive (hot standby)

2008-03-29T21:29:00.000+05:30

This is usually the simplest configuration. Here, you have all your active services (e.g. Apache, MySQL, whatever...) all running on one node. The other node sits doing nothing. (See also A Basic Single IP Address Configuration)

In this case, you can possibly get away with just one DRBD share (resource) for multiple applications. The basic idea is that you have a DRBD share on which you put all your data (not normally binaries or other system files). Let's assume Apache, for example. You might opt for a 'high availability' mountpoint /ha/web. This will be shared across both nodes with DRBD but remember that only the primary node can mount it; you can't have it mounted on both nodes at the same time.

A typical, simple DRBD config for this would look like:

TODO: typical active/passive DRBD config

You would configure Apache with all the websites having document roots under this share, e.g. you might have:

|-/ha
|--- web
|----- client1
|----- client2

snippet from Apache config:

ServerName client1.example.com
DocumentRoot /ha/web/client1
...

ServerName client2.example.com
DocumentRoot /ha/web/client2
...

Setting up heartbeat isn't difficult but there are two steps you need to be aware of:

running the 'drbddisk' script, which triggers DRBD to make the current node primary for the given resource

actually mounting the filesystem
Here's a typical, simple, snippet from haresources showing an active/passive configuration. In this case, node1 is the default primary, and there is a "floating" IP address 192.168.0.1:

node1 drbddisk::web Filesystem::/dev/drbd0::/mnt/ha::ext3 192.168.0.1 httpd

Note the order in which the resources are started. drbddisk needs to go first, since it configures DRBD to be primary. Mounting the filesystem is next, and actually running Apache is last. Setting up the network IP obviously needs to happen at some point before Apache starts.

drbddisk was datadisk in drbd-0.6.x.

web above (the parameter to datadisk or drbddisk) is the resource name you chose for the resource section in drbd.conf, not the device (unless you chose to give your resources device names...)

With several resources, it has to be written like

castor 10.0.0.30 drbddisk::r0 drbddisk::r1 \
Filesystem::/dev/drbd0::/crypt::xfs \
Filesystem::/dev/drbd1::/data::xfs \
samba nfs-kernel-server

Shared Disk in HA evironment

2008-03-29T21:26:00.000+05:30

Shared disk is simply disk which is accessible from more than one computer at the same time.

Commonly these arrangements are fiber channel, and can include redundant FC interconnect cards, and fiber channel switches, so that they can be made with no single point of failure.

Examples include IBM's FAStT(DS4300) and ESS (Shark) lines of fiber channel disks.

Many other manufacturers also make similar products.

The only constraint for this kind of "simple" configuration is that they be accessible from either side without any kind of special command having to be issued other than a mount command

Two Apache Web Servers in an Active/Active Configuration

2008-03-29T21:25:00.000+05:30

Two Apache Web Servers in an Active/Active ConfigurationThis configuration is for a high-availability server which provides two IP addresses (1.2.3.4, and 1.2.3.5) to be failed over between the nodes of our cluster, and an Apache server for each IP address. We will set this up as an active/active configuration.

/etc/ha.d/ha.cf file
logfacility daemon # Log to syslog as facility "daemon"
node paul silas # List our cluster members
keepalive 1 # Send one heartbeat each second
warntime 3 # Warn when heartbeats are late
deadtime 10 # Declare nodes dead after 10 seconds
bcast eth0 eth1 # Broadcast heartbeats on eth0 and eth1 interfaces
ping 1.2.3.254 # Ping our router to monitor ethernet connectivity
auto_failback yes # Keep resources on their "preferred" hosts - needed for active/active
respawn hacluster /usr/lib/heartbeat/ipfail # Failover on network failures

See the ipfail page for more information on ipfail.

/etc/ha.d/haresources file
paul 1.2.3.4 apache::/apache1dir/httpd.cf
silas 1.2.3.5 apache::/apache2dir/httpd.cf

The first word (paul or silas) on the line represents the "preferred" host for the service. The remainder of the line is the list of resources (services) which are part of this ResourceGroup. In this case, each ResourceGroup consists of only two resources -- an IP address, and the apache web server. 1.2.3.4 is a shorthand notation for IPaddr::1.2.3.4, and 1.2.3.5 is a similar shorthand for IPaddr::1.2.3.5.

Because auto_failback was enabled, when paul joins the cluster it will regain the 1.2.3.4 address. Similarly, when silas joins the cluster, it will regain its (1.2.3.5) service address. If an active/passive configuration is desired, then simply change auto_failback to no.

The apache resource agent which comes with Heartbeat supports starting multiple instances of Apache by command-line parameters. The parameter which we've given it tells it where to find its configuration file. So, for our example, we've put the files for the 1.2.3.4 server (normally paul) under /apache1dir, and those for the 1.2.3.5 server (normally silas) under /apache2dir.

/etc/ha.d/authkeys file/etc/ha.d/authkeys must be mode 600. See the section on GeneratingAuthkeysAutomatically for information how to generate good keys automatically.

auth 1
1 sha1 PutYourSuperSecretKeyHere

Apache DirectivesTo get the different Apache instances to bind to the correct IP addresses, you have to tell them which IP addresses they should bind to.

This is done by the Apache Listen directive. In /apache1dir/httpd.cf, this directive should be included

Listen 1.2.3.4:port-number

In /apache2dir/httpd.cf, this directive should be included

Listen 1.2.3.5:port-number

Init DirectivesIt is important that you not let Apache be started by init at boot time. If you do that, then both init and Heartbeat will "fight" for control of Apache, and it won't work. You have to let Heartbeat control all resources that you include in haresources. To disable Apache from starting at boot time, issue the following command on both paul and silas:

/sbin/chkconfig apache off

or if you're using the httpd service script instead of the apache script:

/sbin/chkconfig httpd off

Apache Web Server HA Configuration

2008-03-29T21:24:00.000+05:30

A Simple Apache Web Server HA ConfigurationA common configuration for an HA server is simply to provide an IP address and a single service to be failed over. This example will be an active/passive configuration for the Apache web server.

/etc/ha.d/ha.cf file
logfacility daemon # Log to syslog as facility "daemon"
node paul silas # List our cluster members
keepalive 1 # Send one heartbeat each second
deadtime 10 # Declare nodes dead after 10 seconds
bcast eth0 eth1 # Broadcast heartbeats on eth0 and eth1 interfaces
ping 1.2.3.254 # Ping our router to monitor ethernet connectivity
auto_failback no # Don't fail back to paul automatically
respawn hacluster /usr/lib/heartbeat/ipfail # Failover on network failures

See the ipfail page for more information on the respawn directive above.

In most cases, this file can be identical between the two machines.

/etc/ha.d/haresources file
paul 1.2.3.4 apache

The first word (paul) on the line represents the "preferred" host for the service. The remainder of the line is the list of resources (services) which are part of this ResourceGroup. In this example, the sole resource group consists of two resources: an IP address (IPaddr::1.2.3.4) and the Apache web server. Although Heartbeat can use resources which are simply init scripts from /etc/init.d, both of these ResourceAgents are located in /etc/ha.d/resource.d. In every case, this file must be identical on both machines.

/etc/ha.d/authkeys file/etc/ha.d/authkeys must be mode 600. See the section on GeneratingAuthkeysAutomatically for information how to generate good keys automatically.

auth 1
1 sha1 PutYourSuperSecretKeyHere

Except when changing keys, this file must be identical on the two machines.

Apache DirectivesTo get Apache to bind to the correct IP addresses, you have to tell it which IP address it should bind to.

This is done by the Apache Listen directive. In httpd.cf, include this directive:

Listen 1.2.3.4:port-number

Init DirectivesIt is important that you not let Apache be started by init at boot time. If you do that, then both init and Heartbeat will "fight" for control of Apache, and it won't work. You have to let Heartbeat control all resources that you include in haresources. To disable Apache from starting at boot time, issue the following command on both paul and silas:

/sbin/chkconfig apache off

or if you're using the httpd service script instead of the apache script:

/sbin/chkconfig httpd off

Special ConsiderationsFor the purposes of this example, we assume that somehow the Apache configuration files, and the web site content is being "magically" maintained on both machines and are sufficiently similar that no one will complain when failovers occur between nodes. You can use rsync for this if that meets your needs. Alternatively, one can use shared disk or DRBD when they need to be truly identical to the millisecond.

Clusters

2008-03-29T21:14:00.000+05:30

The High Availability Linux Project

The basic goal of the High Availability Linux project is to:

Provide a high availability (clustering) solution for Linux which promotes reliability, availability, and serviceability (RAS) through a community development effort.

The Linux-HA project is a widely used and important component in many interesting High Availability solutions, and ranks as among the best HA software packages for any platform. We estimate that we currently have more than thirty thousand installations up in mission-critical uses in the real world since 1999. Interest in this project continues to grow. These web pages are average nearly 20000 hits per day, and we see more than 100 downloads of Heartbeat per day.

Heartbeat now ships as part of SUSE Linux, Mandriva Linux, Debian GNU/Linux, Ubuntu Linux, Red Flag Linux, and Gentoo Linux. Ultra Monkey, and several company's embedded systems are also based on it. Although this is called the Linux-HA project, the software is highly portable and runs on FreeBSD, Solaris, and OpenBSD, even on !MacOS/X from time to time.

There have been many articles and several chapters in books written on this project and software. See the PressRoom for more details.

We are now competitive with commercial systems similar to those described in D. H. Brown's 1998 or March 2000 analysis of RAS cluster features and functions. This release 2 series brings technologies and basic capabilities which match or exceed the capabilities of many commercial HA systems. We think you'll be surprised. An R2 getting started guide is available.

We include advanced integration with the DRBD real-time disk replication software, and also work well with the LVS (Linux Virtual Server) project. We expect to continue to collaborate with them in the future, since our goals are complementary.

We have a page of reference sites to provide a few real-life examples of how organizations both small and large use Heartbeat in production. Submissions for this page are actively encouraged.

Heartbeat is a leading implementor of the Open Cluster Framework (OCF) standard.

What Linux-HA can do nowHeartbeat currently supports a very sophisticated dependency model for n-node clusters. It is both extremely useful and quite stable at this point in time. The following types of applications are typical:

Database servers
ERP applications
Web servers
LVS director (load balancer) servers

Mail servers
Firewalls
File servers
DNS servers
DHCP servers
Proxy Caching servers
Custom applications
etc.
Heartbeat is used in virtually every market segment, industry, and organization size.

Heartbeat

The Heartbeat program is one of the core components of the Linux-HA (High-Availability Linux) project. Heartbeat is highly portable, and runs on every known Linux platform, and also on FreeBSD and Solaris. Ports to other OSes are also in progress.

Heartbeat is the first piece of software which was written for the Linux-HA project. It performs death-of-node detection, communications and cluster management in one process.

Sample Cluster Configuration

In these examples, the server names for our cluster will be paul and silas. The cluster is assumed to send heartbeats on both the eth0 and eth1 ethernet interfaces. The IP addresses which will be used as ServiceAddresses to run services on will be 1.2.3.4 and 1.2.3.5, which are on the 1.2.3.0/24 subnet with the default route pointing to 1.2.3.254.

a)A Basic Single IP address Configuration

The most common, basic configuration is that of a high-availability server which simply provides a single IP address (1.2.3.4) to be failed over between the nodes of our cluster. This is an ActivePassive configuration - the most basic configuration.

/etc/ha.d/ha.cf fileThis is for Heartbeat 1.2.x

logfacility daemon # Log to syslog as facility "daemon"
node paul silas # List our cluster members
keepalive 1 # Send one heartbeat each second
deadtime 10 # Declare nodes dead after 10 seconds
bcast eth0 eth1 # Broadcast heartbeats on eth0 and eth1 interfaces
ping 1.2.3.254 # Ping our router to monitor ethernet connectivity
auto_failback no # Don't fail back to paul automatically
respawn hacluster /usr/lib/heartbeat/ipfail # Failover on network failures

This is for Heartbeat 2.0.x without CRM

logfacility daemon
keepalive 1
deadtime 10
warntime 5
initdead 120 # depend on your hardware
udpport 694
ping 1.2.3.254
bcast eth0
auto_failback off
node paul
node silas
respawn hacluster /usr/lib/heartbeat/ipfail
use_logd yes

This is for Heartbeat 2.0.x with CRM

logfacility daemon
keepalive 1
deadtime 10
warntime 5
initdead 120 # depend on your hardware
udpport 694
ping 1.2.3.254
bcast eth0
auto_failback off
node paul
node silas
use_logd yes
compression bz2
compression_threshold 2
crm yes

See the ipfail page for more information on ipfail.

bcast / mcast / ucastIf you want less broadcast traffic, use ucast, which is strictly peer-to-peer. bcast is limited to the logical segment and not routed, while ucast/mcast are potentially routed. ucast duplicates the packets, as it has to be sent to each node and not just broad/multicasted to all of them at the same time.

/etc/ha.d/haresources fileFor Heartbeat version 2 with CRM, you'll need to modify cib.xml instead of this file. Please see the Basic Single IP address Configuration for version 2 page for details.

paul 1.2.3.4

The first word (paul) on the line represents the "preferred" host for the service. The remainder of the line is the list of resources (services) which are part of this ResourceGroup. In this case, there is only one resource -- an IP address. This is a shorthand notation for IPaddr::1.2.3.4. There are many possible variants of how to specify the IP address, to learn about them, see the page on the IPaddr resource agent.

Note that this address cannot be used for anything else on these machines. In particular, it has to be controlled only by Heartbeat, and cannot be brought up by your operating system at boot time. We call this address a ServiceAddress - which is distinct from an AdministrativeAddress, like those brought up by your operating system.

/etc/ha.d/authkeys file/etc/ha.d/authkeys must be mode 600. See the section on GeneratingAuthkeysAutomatically for information how to generate good keys automatically.

auth 1
1 sha1 PutYourSuperSecretKeyHere

b) An Active/Active Two IP address Configuration

A common configuration is that of a high-availability server which simply provides two IP addresses (1.2.3.4, and 1.2.3.5) to be failed over between the nodes of our cluster. We will set this up as an active/active configuration.

/etc/ha.d/ha.cf file
logfacility daemon # Log to syslog as facility "daemon"
node paul silas # List our cluster members
keepalive 1 # Send one heartbeat each second
deadtime 10 # Declare nodes dead after 10 seconds
bcast eth0 eth1 # Broadcast heartbeats on eth0 and eth1 interfaces
ping 1.2.3.254 # Ping our router to monitor ethernet connectivity
auto_failback yes # Try and keep resources on their "preferred" hosts
respawn hacluster /usr/lib/heartbeat/ipfail # Failover on network failures

See the ipfail page for more information on ipfail.

/etc/ha.d/haresources file
paul 1.2.3.4
silas 1.2.3.5

The first word (paul or silas) on the line represents the "preferred" host for the service. The remainder of the line is the list of resources (services) which are part of this ResourceGroup. In this case, each ResourceGroup consists of only one resource -- an IP address. 1.2.3.4 is a shorthand notation for IPaddr::1.2.3.4, and 1.2.3.5 is a similar shorthand for IPaddr::1.2.3.5.

Because auto_failback was enabled, when paul joins the cluster it will regain the 1.2.3.4 address. Similarly, when silas joins the cluster, it will regain its (1.2.3.5) service address. If an active/passive configuration is desired, then simply change auto_failback to no.

/etc/ha.d/authkeys file/etc/ha.d/authkeys must be mode 600. See the section on GeneratingAuthkeysAutomatically for information how to generate good keys automatically.

auth 1
1 sha1 PutYourSuperSecretKeyHere

DNS Architecture

2008-03-29T05:41:00.000+05:30

====================
DNS Architecture
Domain Name Service
====================

[i] /etc/hosts [HOSTS.TXT]
----------------------
Users like names , NWs like numbers

Also called Local resolver or local DNS file

Use of Aliases - historic, generic [www,mail], alternate names [spellings etc]

[ii] Uses of /etc/hosts
-----------------------

* Primary names are used by system to do reverse lookups [ IP - names ]
* Reverse lookups are reqd to create more readable displays

# netstat --inet [-net]

Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 ganesh.bom.labs.n:32822 ganesh.bom.labs.net:ftp ESTABLISHED
tcp 0 0 ganesh.bom.labs.net:ftp ganesh.bom.labs.n:32822 ESTABLISHED
tcp 0 0 localhost.localdom:smtp localhost.localdo:32823 TIME_WAIT
tcp 0 0 localhost.localdom:smtp localhost.localdo:32824 TIME_WAIT

Displays open TCP/IP connections and hosts/Ports involved in the connection

Why use /etc/hosts when we have DNS ?

A: DNS may not be available at boot time, immediately

[iii] Limitations of /etc/hosts
-------------------------------

* Flat file, easy to read/edit, hard to search
* not indexed or encrypted
* Central maint. reqd for new entries [NIC - HOSTS.TXT]
* Govt agency
* Enter a whale of new m/c entries everyday
* Download daily for latest version from NIC so traffic-problems

================
B. DNS Hierarchy
================
[i] Structure of DNS
--------------------

A Unix FS v/s DNS Database Comparision study
=============================================

* Distributed, hierarchical DB
* Localized, not centralized maint. reqd

eg / etc / httpd / conf / httpd.conf <--------- filename
/etc / httpd / remote / httpd.conf
|
root of FS
----------------------->
<--- generic [/] to specific [file] --->

A hostname on the Internet is exactly the opposite

eg willy.dolphin.mammals.org. <-------- dot denotes root of DNS tree
| | | |
hostname sub-domain domain TLD
<-----------------------
<--- specific [host] to generic [.] --->

willy - Name of the computer [hostname -s]
dolphin - sub-domain under 'mammals'
mammals - Domain we have purchased under 'org'
org - A TLD
. - Root node

. or root node is like /, the root of the FS

Dirs are like domains or more precisely sub-domains

Files are like hosts - or computers with IPs and hostnames

Each domain can be further divided or partitioned in to subdomains, just
like a dir can be further subdivided into subdirs.

Directores use '/' as the seperator. Domains are seperated by '.'.

Note:
Subdomains are like dirs unders a parent dir but this dir is not any
normal dir but appears as a dir bcos it is like a NFS mounted share.
A dir which is a FS system on some other host. Can be detached but once
mounted on a particular FS, becomes part of that.

Like every dir, a domain name has a unique name and identifies its
position in the db; Much as a dir's Abs-PN identifies its place in the
FS.
---------------------------------------------------------------------
A domain is a sequence of labels from the node at the root of the domain
to the root of the whole tree, with the labels separated by dots.

In Unix, a dir's abs PN is a list of relative names read from root '/' to
leaf [opp dir to DNS], using a slash to seperate the names.

Just as one can have two files with the same name in seperate dirs,
so can one have two similar hosts but in different domains [nodes].

---------------------------------------------------------------------
DNS requires that sibling nodes - nodes that are children of the same
parent - have different labels - This is to ensure uniqueness.

eg willy.dolphin.mammals.org
willy.whale.mammals.org

Here the parent is 'mammals' and has 2 siblings - 'dolphin' and
'whale'. These have to have different labels.

The above 2 are names of 2 machines i.e. they are 2 completely
different hosts.

Now examine the similarity with the Unix FS :

Similarly, the Unix FS, requires that sibling dirs or files in the same
dir have different names. This is to ensure uniqueness.

Like /usr/local/funny and /usr/bin/funny

Here the parent is '/usr' and has 2 siblings - 'local' and
'bin'. These have to have different labels.

The above 2 are names of 2 subdirs i.e. they are 2 completely
different directories and subsequently 2 different files although
they have the same name.

---------------------------------------------------------------------

Domains and Domain names
------------------------

Domains : A subtree of the domain name space
The domain name of a domain is the same as the domain name of
the node at the very top of the domain.

"." [root node]
|
-----------------------------
| | |
edu org com
|
|
------*<---------- mumbai.edu. node
| mumbai | or the domain name of the mumbai.edu domain
| |
| <------------------- mumbai.edu domain [name space]
| |
| |
-----*-------

Check out the comparision with the Unix FS :

"/" [root node]
|
-----------------------------
| | |
usr bin usr
|
|
------*<---------- /usr/bin/ node
| bin | or the dir name of the /usr/bin dir
| |
| <------------------- /usr/bin/ dir space
| |
| |
-----*-------

-------------------------------------------------------------------

A domain name can also be in many domains.

Eg the domain name "maths.mumbai.edu" is a part of the "mumbai.edu"
domain as well as of the "edu." domain which is once again a part of
the . domain.

All DNS servers are in the . domain

"." [root node]
|
-----------------------------
+-------------------+ | |
| edu | org com
| | |
| | |<----------- edu domain
| --------------- |
| | mumbai |<--------------- mumbai.edu domain
| | --------- | |
| | | maths <-----|------------- maths.mumbai.edu domain
| | | bio <-----|------------- bio.mumbai.edu domain
| | --------- | |
| |-----*-------| |
| |
-------------------

SubDomain Delegation
--------------------

One of the main goals of the DNS was to decentralize administration.
How is this done ?

Consider the CEO of a large Corp. How does she delegate responsibility?

CEO [BOSS] [ "." root node or / ]
|
----------------------------------
| | | |
MKTG SALES HRD ACCTS [Depts or gTLDs or TLDirs]
|
| Mr M
-----------------------
| | |
LOCAL INTL PLANNING
| [sub-Depts or sub-domains delegated
| by Mr M to Mr L, Mr I, Mr P]
Mr L Mr I Mr P
|
-----------------------
| | |
RJ TN WB
Mr R Mr T Mr W
---------
| |
KOL DARJEELING
|
Mr WW [WW.KOL.WB.LOCAL.MKTG.CEO]

She breaks up the org into Depts. Each with its own head.

The Head has total responsibility for his Dept.

The Dept is created by the CEO and hence cannot be made without the CEO
knowing about it.

That is, its made by the CEO and total responsibility is delegated to the
Dept Head to handle his Dept.

The Dept Head CAN create more sub-Depts under his Dept, without consulting
the CEO. He has total authority over his Dept [domain]
He is said to be authorative over his Dept [domain]

CEO only knows about Depts but nothing about the sub-Depts [sub-domains].

Why ?

She does not have to.

That would mean redundancy of information [extra work].
All she has to know is the Dept Head and she can procure whatever info she
needs about sub-Depts etc from the respective Dept Head concerned.

After all, what is he being paid for if she has to keep all his sub-Dept
information !! And all other sub-Depts too.

This is called "Department Delegation".

The parent Dept [domain] retains only pointers to sources of the
sub-Depts [sub-domains] data, so that it can refer queries there.

Now what would happen if I asked the CEO his Question ?

Where is Mr W or WB.LOCAL.MKTG.CEO ?

The CEO would not directly answer this query.

It would refer me to MKTG.CEO. i.e. to Mr M

I would then have to ask [Mr M] MKTG.CEO which would refer me to
LOCAL.MKTG.CEO. [Mr L]

I would then have to ask Mr L who would finally direct me to Mr W at
WB.LOCAL.MKTG.CEO

I have found my man !

This referral business is called 'recursive querying' and most DNS
servers are non-recursive , in that, they just put you on the path to
another server.

----------------------------------------------------------------------

NOW LET'S SEE HOW DNS DOES DOMAIN DELEGATION

"." [root node]
|
-----------------------------
| | |
edu org com
|
-------------------------------
| | -------|---------
kolkatta chennai | mumbai |
| | |
| ------------ |
| | | |
| kalina fort|
| |
-----------------

The "." is controlled by ICANN. [Int'l Corp for Assigned Names/Nos]
"edu" domain is controlled by Network Solutions.
Network Solutions has sub-domains kolkatta, chennai and mumbai.

Network Solutions can handle all the data there, but why should it ?

It therefore delegates the subdomain "mumbai.edu' to the folks at Mumbai
to manage.

The folks at mumbai have total authority over this domain now and can
create more domains [subdomains et al] and they do.

They create "kalina.mumbai.edu" and "fort.mumbai.edu" and let the folks
at Kalina and Fort handle those domains.

At Kalina Office : shiva.kalina.mumbai.edu
ganesh.kalina.mumbai.edu

Some two hosts at the Kalina office.

At Fort Office : shiva.fort.mumbai.edu
ganesh.fort.mumbai.edu

Some two hosts at the Fort office.

Now I do "ping shiva.fort.mumbai.edu".

The query would go to "." which would direct me to a server which handles
the "edu" domain. The "edu" domain knows about the "mumbai" subdomain, so
it directs me another DNS server which knows all about the "mumbai.edu"
domain. Once there, this server again directs me to another DNS server
which know all about "fort.mumbai.edu" domains.

At the DNS server which handles the "fort.mumbai.edu" domain, I finally
find a RR [Resource Record] which is something like this :

shiva.fort.mumbai.edu. IN A 192.168.0.10

And at last, I have the DNS-resolved IP of my shiva.fort.mumbai.edu !

The programs which store info about the domain name space are called
"name servers" and, yes, one of these prgs was running on this last
machine which solved by problem.

Let's do some light stuff now !

[ii] gTLDS : Global Top-Level Domains
---------------------------------------

* com, .edu, .gov, .int, .mil, .net, .org [in US]

DNS Database :
------------- ICANN - [Int'l Corp for Assigned Names/Nos]
Responsible for managing domain name
space

"." [root node] Managed by ICANN
|
-------------------------------------------------------------------
| | | | | | |
org int gov edu mil net com
--------- [Int' orgs]
| |
fish mammals
| | |
---------------
| | |
dolphin whale bat
| | |
willy hector blindy

Hence, willy.dolphin.mammals.org is a host on the DNS DB system

Consider the similarity to the Unix FS :

/
|
-------------------------------------------------------------------
| | | | | | |
etc usr lib int mil net org
--------
| |
httpd
|
conf
|
--------
| |
httpd.conf src.conf

Geopolitical domains : ccTLDs or ISO country codes - eg .uk, .de, .in

2nd Level registration under ccTLDS : eg amazon.de, amazon.co.uk

3nd level domains as states of the US : .ca.us, .ny.us

Eg: 'mammals' is the 2nd level domain purchased and is registered under
the gTLD - 'net'

[iii] Reverse Domains :
-----------------------

* Addr to Name conversion
* Reverse lookups are reqd to create more readable o/p displays / logs etc
* They are called RDs, since they are written in the reverse order

Consider : /etc/hosts

192.168.0.10 crow.birds.org
-----------------------------------------------
Consider the hostname : --------------- [A]

crow .birds. org

<-------- specific [host] ----------> generic [.]
host domain gTLD

-----------------------------------------------

Now consider the IP : --------------- [B]

192.168.0. 10
<--NW portion-->

generic ---------> specific

-----------------------------------------------

Bcos the structure of an IP addr [B] is the opposite of the domainname [A],
to create a 'Reverse Domain Name' we reverse the IP address

10.0.168.192.in-addr.arpa.

'in-addr.arpa.' is a special TLD domain in which all reverse domains are
located.

. [Reverse Domain root DNS servers]
|
arpa
|
in-addr
-------------------------------------------------------------------
| | | | | | |
192.168
|
-------
|->|->|
0 1 255
|
-------
| | |
1->40 254

SSH

2008-03-29T05:36:00.000+05:30

OpenSSH Open Secure Shell
supports both : SSH1 [rsa] / SSH2 [dsa] protocols
OpenSSH suite :
1. sshd - server
2. ssh - client
3. ssh-keygen
4. ssh-agent
5. ssh-add
6. sftp-server
7. sftp
8. scp
============================
* Replacement for the dangerously insecure Berkely r cmds : rlogin, rcp and rsh
* Whenever you telnet or FTP (not anonymous FTP) to a remote machine, you are
asked to identify yourself by giving your login name and a password.
=========================
Case I - without any keys
=========================
Server
[ganesh]
======
1. # adduser foo
passwd ....
2. # service sshd restart
You OpenSSH server is now up
Client
[brahma]
======
1. login as 'foo'
2. # ssh -l foo ganesh
* The first time around it will ask you if you wish to add the remote host
ganesh to a list of "known_hosts"; go ahead and say yes. After saying yes,
and if there is an account on the remote system ganesh named foo, it will
prompt you for foo's password on the remote system [ganesh].
* If you type in the remote password correctly it should let you in.
3. Type 'yes'
4. Put you password which you gave on the remote SSH server for foo [a VLU]
* You are remotely logged in with ssh on ganesh. Just like telnet but secure.
* Examine your home on brahma [client] : /home/foo/.ssh/known_hosts
Will contain the sshd server name and a DSA algol key.
* The server you just successfully connected to is now a _known_host_
[sshd server ganesh is now a known host server]
* The next time you ssh to ganesh, this file is verified and now you will only
be asked for a password.
* The 'mitm' [man-in-the-middle] attack could still trip you dangerously
===================
Case II - with keys
===================
Client
[brahma]
==========
1. Login as foo
2. ssh-keygen -d
or -t rsa <--- to generate a rsa key
* Will autom generate a DSA public-private key pair of 1,024 bits.
* Press CR for the default file name to save the keys in :
/home/foo/.ssh/id_dsa
* Subsequently, you'll be asked for a passphrase (password) with confirmation.
* This can be anything and can be very long. Over 600 characters will be
accepted in the passphrase.
* The point is that passphrases are not supposed to be like your normal
password and should be much longer and kept just as secret as your normal
password.
* After you've typed in a passphrase twice it will spit out a long string of
numbers with your username@hostaddress at the end. Should look something
like this:
Your public key is:
ssh-dss AAAAB3NzaC1kc3MAAACBANTlkP9edHvaSnr+QDxD3cdJFOpOiGHTultOlp5J7RWzpENIb/AnK0iuYubCErYgGDiyqzR4duXD4V+zNy0vI1++pdFjpkN1h0X1RaCaJ04SnSca7jfnG7Pt65sYNoTldNl20RUZBmb/NbBttWBvLJxsGGs0pawYxXFSlhI7COhBAAAAFQCRW47ddyLK71XJu4vEY38OjpxAvwAAAIBCC+Ki98tAtmwFL+Sdvxxr1HA3bHQSqBUxOZawOkB7ri0l93o/wMcl8e20N+ffrGnVyc7DgI8klt2qXrJQ9LBasLl9aJR+HfUzOGfA8lSdJlxgAfFxHzOXVqgQ7OXhYKsHCs6MwixsuxSH0Nuaw2x4ZLs0ti09KJPwoL0gF8ptNgAAAIBA4MYs/MbtI1dkzk2ltq+vyOVVDuLdhTrMdCfLHGJWUij6wgbCe7wGwDOIR493LrUkgJkB01qru1qHRr1WAzlR/ZZ2Sv0CVsrRCBFN3etrMWhICl/1wV/3Cp7hev1gfjNY2GgXIo51ujIU/yFgQna83Paf2s43e9l7MmB5LdwctA== bar@brahma.bom.labs.net
* However your terminal program will have wrapped the long lines around so
that it all fits on one screen. It's important to keep in mind though that
the string of numbers is and must all be on one line as it is shown above.
* The big line above consists of 4 sections. It's important that each of
those sections have no spaces in them. The first section is the bit level
and is usually going to be 1024, the second section is the salt, the third
is the actual key and the forth section is the user@host identification.
* You will be asked this paraphrase whenever you access the sshd machine.
* You will notice a new dir created in your home : /home/foo/.ssh/
The public and private keys just generated stored viz :
/home/foo/.ssh/
---------------
private key : /home/foo/.ssh/id_dsa
public key : /home/foo/.ssh/id_dsa.pub
* That's all on the client side !
Server-side Config
[ganesh]
==================
1. Login as root.
2. # service sshd start
3. # su - 'foo'
4. # mkdir .ssh
5. # cd /home/foo/.ssh/
6. Copy & Rename the file : /home/foo/.ssh/id_dsa.pub from 'brahma' [Client]
to /home/foo/.ssh/authorized_keys on 'ganesh' using scp or ftp or whatever!
Lets see the contents of foo on brahma w/o logging in
a. # ssh brahma ls
Now lets copy the file /home/foo/.ssh/id_dsa.pub from 'brahma'
here and rename it as authorized_keys
b. scp brahma:.ssh/id_dsa.pub authorized_keys
7. Make sure the perms of /home/foo/.ssh/authorized_keys file on server 'ganesh'
is 600 i.e. R/W only by foo
# chmod 600 authorized_keys
8. Make sure dir /home/foo/.ssh/ has perms of 700
chmod 700 /home/foo/.ssh/
Alert !!! ---> Make sure 7 and 8 are satisfied otherwise ssh will not work

Execution :
=========
1. Login from client [brahma] to server [ganesh] as foo.
# ssh -l foo ganesh
2. You will now be prompted for your paraphrase.
* This will be matched with your local [brahma] private key and not the
password of foo on ganesh. [/home/foo/.ssh/id_dsa]
* If it matches a request will be sent to the SSH server running on
"ganesh:22".
* The SSH client on the local machine brahma contacts the sshd daemon on the
remote SSH server [ganesh].
* The client tells the server that a user 'foo' wants to connect.
* The server then examines who made the request - 'foo'
* It will check whether 'foo' is a VLU in /etc/passwd.
* If OK, then it will create a challenge using the public key file you had
copied to foo's home on the server, before, from foo's home on brahma :
/home/foo/.ssh/authorized_keys
Remember we had renamed it ?
* If an entry is there for a public key corresponding to the user, the server
generates a random message, constructs an MD5 message digest and encrypts it
with the user's public key - which is now the above file
* This is then sent to the client [brahma].
* The client looks in this identification file aka private key. Since only
the client has the corresponding private key, which is :
[/home/foo/.ssh/id_dsa] on brahma
it can only decrypt the message. This results in the message digest.
* Now the client computes a checksum of the message digest and sends it back to
the server.
* The server then computes the checksum of the message digest it had sent to the
client. If this checksum and the checksum received from the client are the
same, then the client can be trusted to have the correct private key, and the
authentication procedure is completed.
* You normally give foo's local login passwd on server to access FTP etc.
But here that is never asked.
* Never once are you prompted for any password at all.
* Hence, sshd never uses passwords to login and that is one huge leap
as far as security consciousness is concerned !

======================
Case III
ssh-agent* and ssh-add*
=======================
* How about starting a SSH session, fully secure and never having to enter a
password or even the passphrase too ?
* Well, weird as it sounds security-wise, this is exactly what the ssh-agent
does on the client
* Now try this :
Client
[brahma]
========
1. Login as foo
2. Append the following line to and relogin as foo:
/home/foo./.bash_profile
========================
eval `ssh-agent`
or from the cmd line : # eval $(ssh-agent)
You should now see a message like : Agent pid [number]
* The authentication agent must be running and must be an ancestor of the
current process for ssh-add to work. You need the shell's eval cmd for this.
* ssh-agent executes several commands, creates an PID file and sets some
system env variables.
3. Re-login and the ssh-agent will now be running as 'foo'
4. Now do this :
# ssh-add ----> will load SSH2 keys - The DSA ones
or
# ssh-add .ssh/id_rsa ----> will load SSH1 keys - The RSA ones
Enter the paraphrase you had entered the first time you made the private
keys on the client.
If it matches, you will get the following message :
Identity added : /home/foo/.ssh/id_rsa (/home/foo/.ssh/id_rsa)
5. Try this now :
# ssh -l foo ganesh
And you are in - without password authentication, but totally secure !
* Luckily, ssh-agent doesn't remember the information after you logout.
Otherwise, once a passphrase was entered with ssh-add, the security of a
protected passphrase would be pointless if somebody broke into your system.
* So after you logout, you will have to do a 'ssh-add' for the session
* The agent is only valid for the duration of the session

scp*
Copy files securely

* SSH gives you access to a set of commands and a shell on a remote machine.
By itself, it does not enable you to copy files, it however provides the
'scp'command.
- You are foo, logged in to Client brahma
- You wish to copy a file foo1 from your home on the ssh server
ganesh to your home on the remote client brahma or wherever
you are currently logged in
ganesh brahma
[ssh server] [client]
============= ========
/home/foo : /home/foo :
========== ==========
-> Copy file foo1 from ganesh
# ls # ls
# foo1 foo2 # scp ganesh:foo1 . [dot is mandatory]
# ls
# foo1
-> Copy file /etc/hosts from ganesh
# scp ganesh:/etc/hosts .
# ls
# foo1 hosts
# ls
-> Copy file foo1 to ganesh's /tmp
# scp foo1 ganesh:/tmp

-> Copy file foo3 to foo's home on ganesh
# scp foo3 ganesh:
# cd .ssh/
----> foo's own config file # vi config
for global use Host *gbln
/etc/ssh/ssh_config HostName ganesh.bom.labs.net
User foo
ForwardAgent yes
Now use will be able to do this :
# scp foo3 gbln:
* 'scp' assumes your home directory on the server to be your current working
directory, so if you are using relative paths for the server, they have to
be relative to the location of your home dir on that machine.
* Using the -r switch for 'scp', you can also copy directories recursively.
* 'scp' also allows you to copy files between two remote machines other
than your own, of course.
* Now you might be tempted to try something like this:
You open an SSH connection to ganesh.bom.labs.net. Once you are
logged in, you type
# scp brahma:b1 .
to copy the local file 'b1' to the remote server you are currently logged
in. Most likely you will get a message like :
ssh: secure connection to [brahma machine] refused
What has happened is that you executed the remote version of 'scp', and
it tried to connect to a non-existant SSH server running on your machine
which is brahma !!
MOTS :
Remember to run 'scp' always from a local terminal, unless your machine
[brahma] also runs an SSH server.

sftp*
Copy files securely with sftp

* If you prefer a more 'ftp-ish' approach, try 'sftp'
* 'sftp' establishes an SSH tunneled FTP connection to a server and allows you
to use most of the standard FTP command set.
* Since version 2.0.7, the popular graphical FTP client - gftp - supports
sftp-transfers, which makes up for sftp's somewhat limited feature set.
[we are using gftp-2.0.14-2 in RHL9]
[foo@brahma foo]$ sftp ganesh
You will be asked for the paraphrase :
sftp>
* The sshd server starts up the sftp server auto on connect by the sftp
client.
This sftp-server is : /usr/libexec/openssh/ftp-server
'sftp' will activate this server automatically upon connect, and you'll need
no extra permissions on the remote server.

Configuring the OpenSSH -
Server and Client

Configuring the The OpenSSH client - /usr/bin/ssh

Configuring The Client : Global client config file : /etc/ssh/ssh_config
Per user config file : $HOME/.ssh/config
[by def: dir / file nonexistant]
OpenSSH knows three configuration levels:
* command line options
* User configuration file
* System-wide configuration file ('/etc/ssh/ssh_config').
* Options given on the command line prevail over configuration file options
* Options given in the user's configuration file prevail over those in the
system-wide configuration file.
* All command line options are available as configuration file options.
* Since there is no user config file installed by default and hence, no
$HOME/.ssh/ dir, copy and rename '/etc/ssh/ssh_config' to '~/.ssh/config'
(or edit '/etc/ssh/ssh_config' as 'root').
* The standard config file looks like this:
[lots of explanations and possible options listed]
# Be paranoid by default
Host *
ForwardAgent no
ForwardX11 no
FallBackToRsh no
* The config file is read sequentially, i.e. the first setting that matches a
pattern 'wins'.
* Let's say you have an account at ganesh.bom.labs.net and your account name
is 'foo'.
* Furthermore you want to use the 'ssh-agent' - 'ssh-add' combo (see before) as
well as data compression to speed up transfers.
* And since you are too lazy to type the full hostname every time, you want to
use 'gbln' as an abbreviation for 'ganesh.bom.labs.net'
* Your config file should then look like this:
Host *gbln
HostName ganesh.bom.labs.net
User foo
ForwardAgent yes
Compression yes
# Be paranoid by default
Host *
ForwardAgent no
ForwardX11 no
FallBackToRsh no
* Next time, as foo, you enter :
# ssh gbln
SSH will look up the full hostname, use your user name to login and
authenticate using the key managed by the 'ssh-agent'. It can't get much
easier than that, can it?
* SSH connections to all other hosts will still use the 'paranoid' default
settings, the configured accounts only those paranoid settings which haven't
been explicitly turned off in their configuration or on the command line.
* In the example above, an SSH connection to ganesh.bom.labs.net will have these
options set to 'yes': 'ForwardAgent' and 'Compression', these options,
however, will still be set to 'no' unless overridden by command line arguments
: 'ForwardX11' and 'FallBackToRsh'.
* Further options you might want to have a look at include:

CheckHostIP yes
This option performs an
additional IP address check on the remote host to prevent
DNS spoofing.

CompressionLevel
The compression level
ranges from '1' (fast) to '9' (best). Default is '6'.

ForwardX11 yes
You will need this option
to run remote X applications locally.

LogLevel DEBUG
This option comes in
handy when you've got trouble with your SSH connection. The
default setting is INFO.

===================================================
Configuring the The OpenSSH server - /usr/sbin/sshd
===================================================
* SSH server configuration file : /etc/ssh/sshd_config
* Note that OpenSSH does not have different configuration files for SSH 1.x
and 2.x.
* Among the default options you might want to have a look at, are :
PermitRootLogin yes : A preferable option might be PermitRootLogin
without-password which disables 'root' logins from
machines without a matching key pair.
Setting this option to 'no' disables 'root' logins
completely and you'll have to use su from a user
account.
X11Forwarding no Change this option to 'yes' to allow your users to
run X apps on your machine.
Furthermore, disabling this option doesn't improve
your server's security since users can always
install their own forwarders
PasswordAuthentication yes
Setting this option to 'no' will only allow SSH
logins using the key mechanism. Cleartext passwords
tunnelled thru SSH will not be allowed.
This might annoy users who are logging in from
different machines frequently but is a boost to
server security.
(password-based authentication schemes are weak)
Subsystem sftp /usr/libexec/openssh/sftp-server
This is def ftp server bcos users may not like the
cryptic syntax of scp.
Install the 'sftpserv' rpm, which is a SSH tunneled version
of FTP ('sftpserv' is part of the sftp package).
Since users are familiar with ftp, this pkg is exactly like
that and is a worthwhile thing to install.
Do not forget to change the path above to
'/usr/bin/sftpserv'
after install the new server.
Moreover, the popular graphical "gftp" program supports
transfers via 'sftp' since version 2.0.7 (which makes up for the missing features in pure 'sftp').

LVM

2008-03-29T05:33:00.000+05:30

A.
1. Create a 500 MB part of 8e for /home in fdisk
2. U cannot give any label in DD for this - NA
3. Install minimum
B.
# vgscan -
# vgdisplay - will give "no volume groups found"
C. Create a PV called /dev/hda5
Note: 1. In Linux, one cannot access the HW partition hda5 directly -
you go thru the /dev/hda5
'ly, in LVM, one cannot access the HW partition hda5 directly -
you go thru the PV - /dev/hda5
Since Linux and LVM look and access the partition differently -
2. The devdrv name and the PV name are the same, not surprisingly
3. Hence, what linux calls a devdrv, LVM calls a PV
# pvcreate /dev/hda5 <=============================
# pvscan
# pvdisplay /dev/hda5
D. Create a VG called vadapav
# vgcreate vadapav /dev/hda5
# vgscan -
# vgdisplay -
Note : A /dev/vadapav/ dir is created for every VG
E. Create a LV called idli
# lvcreate -l 120 -n idli vadapav <==========================
or
# lvcreate -L 480 -n idli vadapav
which will create
/dev/vadapav/idli ----- Devdrv file is the LV
F. # mke2fs /dev/vadapav/idli <============================
This will Make a FS on hda5 part. of type ext2
I should have given -j
# mount /dev/vadapav/idli /home
# df -h
# umount /home
# tune2fs -j /dev/vadapav/idli
# mount /home
Now u have an ext3 one
To resize a LV :
e2fsadm -l+60 /dev/vadapav/idli
or
e2fsadm -L+240 /dev/vadapav/idli
Adding another Disk
===================
1. fdisk /dev/hdc
* Create a single primary partition the size of the whole disk
* Change id to 8e
* Save and exit from fdisk
* Re-Read new parition table with :
sfdisk -R /dev/hdc
or
hdparm -z /dev/hdc
or
partprobe
or
plain CAD
2. Create PV : # pvcreate /dev/hdc1
3. Add this new PV to the existing VG - vadapav
# vgextend vadapav /dev/hdc1
4. Now resize the old LV - after umounting, of course -
# e2fsadm -l+xxx /dev/vadapav/idli
5. Now mount /dev/vadapav/idli on /home and you have resize /home to
spread across two disks
Removing a Disk
===============
1. Resize the LV - after umounting, of course -
# e2fsadm -l-xxx /dev/vadapav/idli
2. lvremove /dev/vadapav/idli
3. Now remove the LV from the VG too
# vgreduce vadapav /dev/hdc1
***********************************
Summary
=======
A.
1. Create a 500 MB part of 8e for /home in fdisk
2. U cannot give any label in DD for this - NA
3. Install minimum
B.
# pvcreate /dev/hda5 <=============================
C.
# vgcreate vadapav /dev/hda5
D.
# lvcreate -l 120 -n idli vadapav <==========================
E.
# mke2fs -j /dev/vadapav/idli <========================
F.
# mount /dev/vadapav/idli /home
and
configure fstab
Command Used Summary :
====================
* vgscan - Scan LVM system for all VG
* vgdisplay vadapav - To see a particular VG
* vgchange -a n vadapav - To deactivate a VG
* vgremove vadapav - To remove a VG
* vgextend vadapav /dev/hdaX - To add a PV to the VG
* vgreduce vadapav /dev/hdaX - To delete a PV from a VG
* pvscan - Scan LVM system for all PVs
* pvdisplay /dev/hdaX - To see a particular PV
* lvscan - Scan LVM system for all LVs
* lvdisplay /dev/vadapav/idli - To see a particular LV
* lvremove /dev/vadapav/idli - To delete a LV
Alert :
Q: Reduce the LV to 200 MB
RHL9: 1. df -h
/dev/vadapav/idli 101M 4.5M 91M 5% /home
2. umount /dev/vadapav/idli
3. e2fsadm -L200 /dev/vadapav/idli
4. mount -a
Q: Extend the LV to 500 MB
AS4: 1. df -h
/dev/vadapav/idli 101M 4.5M 91M 5% /home
2. lvextend -L500 /dev/vadapav/idli
3. ext2online /dev/vadapav/idli

NFS

2008-03-29T05:24:00.000+05:30

Why use NFS ?
Scenario
A small office has an old Linux server that is running out of disk space.
The office can't tolerate any down time; even hours, because the server is
accessed by overseas programmers and clients at nights and local ones by day.
Budgets are tight and the company needs a quick solution until it can get a purchase order approved for a hardware upgrade.
Another Linux server on the network has additional disk capacity in its
/jokes partition and the office would like to expand into it as an
interim expansion NFS server.
For the scenario you need:
Read-only access to the /jokes directory to all networks
Read/write access to the /jokes from all servers on the 192.168.0.0 /24
network, which is all addresses from 192.168.0.0 to 192.168.0.255
In all cases, use the sync option to ensure that file data cached in
memory is automatically written to the disk after the completion of any
disk data copying operation.

Part - I
Server-side NFS Config
1. Which dir do u want to share
2. configure /etc/exports
3. service portmap restart
4. service nfs restart [Use "exportfs -r" to refresh rpc.nfsd which re-reads
/etc/exports - of course, when you modify it only]
5. service nfslock restart

Testing tools :
===============
6. showmount -e [localhost]
7. rpcinfo -p [localhost]
aka
/etc/rc.d/init.d/portmap
portmap* : Port 111 [portmapper]
Note: # ldd `which portmap`
> libwrap.so.0 => /usr/lib/libwrap.so.0 (0x40034000)
Hence one can use TCP wrappers to block portmap ergo nfs from clients
eg /etc/hosts.deny
===============
portmap:192.168.0.30
/etc/rc.d/init.d/nfs
====================
rpc.mountd* : Emphemeral Ports provided by portmap [mountd] Remote Mounting
rpc.nfsd* : Port 2049 This is the NFS server [nfsd] Big Boss
rpc.rquotad* : Emphemeral Ports provided by portmap [rquotad] NFS Quotas
/etc/rc.d/init.d/nfslock
========================
rpc.lockd* : Emphemeral Ports provided by portmap [nlockmgr] Concurrency Ctl
rpc.statd* : ------- do ------- [status] NFS status
8. nfsstat -s <-------- Server stats : see that bad calls is < 3%
9. nfsstat -c <-------- Clients stats : see that bad calls is < 3%
10 nfsstat -a <--------

Part - II
Client-side NFS Config

1. service portmap restart
Always start this first if 2 m/cs have to share files
2. service nfslock restart
3 Find out what the NFS server is sharing :
showmount -e NFSServer
4. mkdir /funny Default: is UDP
5. mount ganesh:/jokes /
mount ganesh:/jokes /funny -o soft,intr,rsize=8192,wsize=8192 tcp
\6. cd /funny
7. ls

/etc/exports
============
/jokes *.bom.labs.net(rw)
Note : NFS uses UDP but one can use TCP too if the kernel supports it

Automation of NFS Server and Client
using fstab and automount*
Using fstab
===========
Server :
======
1. Disable FWs
2. which dir do u want to share
3. configure /etc/exports
==> /jokes
4. Using chkconfig enable portmap, nfs and nfslock
Client :
======
1. Configure /etc/fstab
====> 192.168.0.20:/jokes /funny nfs soft,intr 0 0
2. Using chkconfig see that portmap, nfslock and netfs are set on
3. On reboot, /etc/rc.d/rc.sysinit will automount
Using autofs
============
Example I
=========
Server :
======
Same as above
Client :
======
1. # mkdir -p /export/nfs/
2. Configure /etc/auto.master
====> /export /etc/auto.misc --timeout=60
3. Configure /etc/auto.misc
key

\/
====> nfs 192.168.0.20:/jokes
or
====> nfs -fstype=nfs,soft,intr,rsize=8192,wsize=8192,nosuid,tcp \
192.168.0.20:/jokes
4. service autofs restart
5. cd /export/nfs/
and you will see the contents of /jokes/ dir on nfs server ganesh
6. Using chkconfig see that portmap, nfslock and autofs are set on
On reboot, the autofs script will start automount*
Example II
==========
Server :
======
1. Disable FWs
2. which dir do u want to share
3. configure /etc/exports
==> /home *(rw)
4. Using chkconfig enable portmap, nfs and nfslock
Client :
======
1. Configure /etc/auto.master
====> /home /etc/auto.misc --timeout=60
2. Configure /etc/auto.misc
key

\/
====> ajay 192.168.0.20:/home/ajay
or
====> ajay 192.168.0.20:&
3. service autofs restart
4. Now log in as ajay
and you will see the contents of /home/ajay/, Ajay's home dir
on the NFS server
Here, the key is ajay, so the ampersand wildcard is interpreted to mean
ajay. This means you'll be mounting the ganesh:/home/ajay directory
from the NFS server on ajay's home dir here, on the Client, auto
5. Using chkconfig see that portmap, nfslock and autofs are set on

Part - IV
NFS log Files

/var/lib/nfs/
=============
etab : export perms of /jokes as shared with all users
====
/jokes (ro,sync,wdelay,hide,secure,root_squash,no_all_squash,subtree_check,secure_locks,mapping=identity,anonuid=-2,anongid=-2)
rmtab : which hosts have mounted your shares
=====
brahma.bom.labs.net:/jokes:0x00000001
shiva.bom.labs.net:/jokes:0x00000002
xtab : per host
====
/jokes brahma.bom.labs.net(ro,sync,wdelay,hide,secure,root_squash,no_all_squash,subtree_check,secure_locks,mapping=identity,anonuid=-2,anongid=-2)
/jokes shiva.bom.labs.net(ro,sync,wdelay,hide,secure,root_squash,no_all_squash,subtree_check,secure_locks,mapping=identity,anonuid=-2,anongid=-2)

Part - V
NFS Tuning
================================================================================
* rsize and wsize are defaulted to 4K in Linux. Increase to 8k
* For TCP make it 32K. UDP 8K is OK
* In 2.4 kernels, the default input queue is 64
# cat /proc/sys/net/core/rmem_default ----> 65535
# cat /proc/sys/net/core/rmem_max ----> 65535
Increase this to 256K, a good default
# echo 262144 > cat /proc/sys/net/core/rmem_default ----> Now 256K
# echo 262144 > cat /proc/sys/net/core/rmem_max ----> Now 256K
Now restart nfs : service nfs restart which is now tuned
Replace the defaults back to their original values so as not to negatively
affect other daemons
# echo 65535 > /proc/sys/net/core/rmem_default
# echo 65535 > /proc/sys/net/core/rmem_max
Additional use "procinfo*" to see kernel details

Part - VI
NFS Security

* NFS and portmap have had a number of known security deficiencies in
the past. As a result, its not recommended using NFS over insecure
networks.
* NFS doesn't encrypt data and it is possible for root users on NFS clients
to have root access the server's filesystems.
* You can exercise security-related caution with NFS by following a few
guidelines:
o Restrict its use to secure networks
o Export only the most needed data
o Consider using read-only exports whenever data updates aren't
necessary.
o Use the root_squash option in /etc/exports file (default) to reduce
the risk of the possibility of a root user on the NFS client having
root file permission access on the NFS server. This is normally an
undesirable condition, especially if the NFS client and NFS server
are being managed by different sets of administrators.
These points should be the foundation of your NFS security policy

Part - VII
NFS Hanging / TroubleShooting

* As seen before, if the NFS server fails, the NFS client waits indefinitely
for it to return. This also forces programs relying on the same client
server relationship to wait indefinitely too.
* For this reason, use the soft option in the NFS client's /etc/fstab
file. This causes NFS to report an I/O error to the calling program
after a long timeout.
You can reduce the risk of NFS hanging by taking a number of
precautions:
o Run NFS on a reliable network
o Avoid having NFS servers that NFS mount each other's filesystems or directories.
o Always use the sync option whenever possible.
o Do not have mission-critical computers rely on an NFS server to operate, unless the server's reliability can be guaranteed.
o Do not include NFS-mounted directories as part of your search path, because a hung NFS connection to a directory in your search path could cause your shell to pause at that point in the search path until the NFS session is regained.

VSFTPD

2008-03-29T05:22:00.000+05:30

VSFTPD 1.1 - Very Secure FTP Daemon

Other FTP servers : WU-FTPD, Pro-FTP etc etc

Config files : 1. /etc/vsftpd/vsftpd.conf
/etc/vsftpd.chroot_list
2. /etc/vsftpd_users
3. /etc/vsftpd.user_list
/etc/vsftpd.banned_emails
GUI Client: gftp
CLI Client: ftp
-----------------------
/etc/vsftpd/vsftpd.conf
-----------------------
* vsftpd is PARANIOD! By default it allows almost nothing.
* Note: DO not use a space before or after the '='. It is an error !
VSFTPD WILL REFUSE TO START !
* Note: All directives should start from column 1 or else error !
VSFTPD WILL REFUSE TO START !
* If there is an error in a directive [misspelling], the vsftpd
daemon will fail to start or die on you. So, watch out!
* vsftp has a wealth of config options.
Lets look at some 35 of them.
Three types of FTP users :
1. Real user - in which you see : /home/foo and can go anywhere
and can even DL your passwd file
2. Guest user - in which you see : / and can go nowhere
The user is chroot'd to his home and will always see /
Cannot go anywhere ! Called a FTP chroot jail
Both the above are VLUs
Both will see their homes
3. Anonymous FTP user - Not a VLU
Will be logged in as user : ftp/ftp 14:50
Home will be : /var/ftp/

List of Directives covered :
----------------------------------
A. Directives for anonymous logins
----------------------------------
1 anonymous_enable=YES
2 no_anon_password=NO
3 anon_root=(none)
4 ftp_username=ftp
-----------------------------------------
B. Directives for VLU [Real/Guest] logins
-----------------------------------------
5 local_enable=NO
6 write_enable=NO
7 chroot_local_user=NO
8 local_root=no default
9 chroot_list_enable=NO
10 chroot_list_file=/etc/vsftpd.chroot_list
11 local_max_rate=0 [unlimited]
12 guest_enable=NO
13 guest_username=ftp
---------------------------------
C. Ports / Connections / TimeOuts
---------------------------------
14 listen=no
15 listen_port=21
16 listen_address=default none
17 tcp_wrappers=NO
18 max_clients=0 [unlimited]
19 max_per_ip=0 [unlimited]
20 data_connection_timeout=300
21 idle_session_timeout=300
-----------------------
D. Banners and Messages
----------------------
22 ftpd_banner=(none - default vsftpd banner is displayed)
23 banner_file=(none) Set to some filename
-----------------------------
E. Fine-Tuning Access Control
-----------------------------
24 userlist_enable=NO
25 userlist_file=/etc/vsftp.user_list
26 userlist_deny=YES
27 user_config_dir=(none) or somedir/
----------
F. Logging
----------
28 xferlog_enable=NO
29 xferlog_file=/var/log/vsftpd.log
30 xferlog_std_format=NO
31 log_ftp_protocol=NO
--------------------
G. Uploads/Downloads
--------------------
32 anon_upload_enable=no
33 ascii_upload_enable=no
34 ascii_download_enable=no
----------------
H. Miscellaneous
----------------
35 pam_service_name=vsftpd
*******************************************************************************
Now lets examine them one by one !
By functionality !
************************
----------------------------------
A. Directives for anonymous logins
----------------------------------
1 anonymous_enable=YES
2 no_anon_password=NO
3 anon_root=(none)
4 ftp_username=ftp
1. anonymous_enable=YES
Allow anon logins or not.
ftp' and 'anonymous' are VLUs for anonymous.
All other anonymous-based directives [which follow] are obviously
rendered useless, if this is set to 'NO'
Def: Yes
2. no_anon_password=NO
If yes, anonymous users can log straight in without a password
3. anon_root=/win Def: none
On anonymous login, vsftpd will change over to this /win directory
instead of the def: /var/ftp/.
Note : If /win is world writeable or not owned by root, then failure!
Other failures are silently ignored.
4. ftp_username=ftp
This is the name of the user we use for handling anonymous FTP.
The home directory of this user is the root of the anonymous FTP area.
-----------------------------------------
B. Directives for VLU [Real/Guest] logins
-----------------------------------------
5 local_enable=NO
6 write_enable=NO
7 chroot_local_user=NO
8 local_root=no default
9 chroot_list_enable=NO
10 chroot_list_file=/etc/vsftpd.chroot_list
11 local_max_rate=0 [unlimited]
12 guest_enable=NO
13 guest_username=ftp
5. local_enable=NO If this is not YES, no VLU allowed in. Server is anon
Controls whether local logins are permitted or not. If enabled, normal user
accounts [Real] in /etc/passwd may be used to log in.
This means, by default, vsftpd only allows anon users in.
Wonderfully Paraniod!
VSFPTD is now an Anonymous FTP server, by default !
6. write_enable=NO
For any type of FTP write command, this should be YES
If enabled, allows file uploads. VLU's and anon ftp users [ftp 14:50]
must,then, have write permission on the upload dir. def: /var/ftp/
7. chroot_local_user=NO
If set to YES, VLUs will be placed in a chroot() jail in their home dir.
after login. BUT....
IMP:
====
If set to YES, and [#18] "chroot_list_enable=YES" which obviously implies
that [#12] "chroot_list_file=/etc/vsftpd.chroot_list" is searched.
Then the meaning is the opposite. Users on this file-list WILL NOT be put
in a chroot [their HOME] jail.
Warning: This option has security implications, especially if the users
have upload permission, or shell access. Only enable if you know what you
are doing.
Note that these security implications are not vsftpd specific. They
apply to all FTP daemons which offer to put local users in chroot() jails.
08 local_root=none
This option represents a directory which vsftpd will try to change to after
a local (i.e. non-anonymous) login. Not chroot'd
Failure is silently ignored.
Similar in function to anon_root [#3]
09 chroot_list_enable=NO
If activated, you may provide a list of local VLUs who are placed in a
chroot() jail in their home directory upon login.
The meaning is slightly different if "chroot_local_user" is set to YES.
IMP: [See #9] above
====
In this case, the list becomes a list of users which are NOT to be placed
in a chroot() jail.
By default, the file containing this list is /etc/vsftpd.chroot_list,
but you may override this with the chroot_list_file setting.
10 chroot_list_file=/etc/vsftpd.chroot_list
The option is the name of a file containing a list of local users which
will be placed in a chroot() jail in their home directory.
This option is only relevant if the option [#17] chroot_list_enable
is enabled, and the option [#9] chroot_local_user is disabled.
11 local_max_rate=0 [unlimited]
The maximum data transfer rate permitted, in bytes per second, for local
authenticated users i.e. VLUs
12 guest_enable=NO Normal VLUs converted to anon users and thrown to
/var/ftp/, but not chrootd, unless chroot_local_user
is given.
If enabled, all non-anonymous [VLUs] logins are classed as "anon" logins
and on # pwd will get /var/ftp/
The login is remapped to the user specified in the 'guest_username'
setting, which is ftp, by default
This is like root_squash in NFS
13 guest_username=ftp
If the [above #15] 'guest_enable' is YES, this setting is the real username
which guest users [formerly VLUs] are mapped to.
---------------------------------
C. Ports / Connections / TimeOuts
---------------------------------
14 listen=no
15 listen_port=21
16 listen_address=default none
17 tcp_wrappers=NO
18 max_clients=0 [unlimited]
19 max_per_ip=0 [unlimited]
20 data_connection_timeout=300
21 idle_session_timeout=300

14 listen=no
If enabled, vsftpd will run in standalone mode.
vsftpd will then take care of listening for and handling incoming
connections on 20/21 ports
15 listen_port=21
If vsftpd is in standalone mode, which is always the case, this is the port
it will listen on for incoming FTP connections.
16 listen_address=default none
To make vsftpd non-promiscuous and listen on only one IP address.
If vsftpd is in standalone mode, the default listen address (of all local
interfaces) may be overridden by this setting.
Provide a numeric IP address.
17 tcp_wrappers=NO
If enabled, and vsftpd was compiled with tcp_wrappers support, incoming
connections will be fed through tcp_wrappers access control. Furthermore,
there is a mechanism for per-IP based configuration. If tcp_wrappers sets
the VSFTPD_LOAD_CONF environment variable, then the vsftpd session will try
and load the vsftpd configuration file specified in this variable.
18 max_clients=0 [unlimited]
If vsftpd is in standalone mode, this is the maximum number of clients which
may be connected. Any additional clients connecting will get an error message.
19 max_per_ip=0 [unlimited]
If vsftpd is in standalone mode, this is the maximum number of clients which
may be connected from the same source internet address. A client will get an
error message if they go over this limit. To prevent DOS.
20 data_connection_timeout=300
The timeout, in seconds, which is roughly the maximum time we permit data
transfers to stall for with no progress. If the timeout triggers, the remote
client is disconnected.
21 idle_session_timeout=300
The timeout, in seconds, which is the maximum time a remote client may
spend between FTP commands. If the timeout triggers, the remote client is
disconnected.
-----------------------
D. Banners and Messages
-----------------------
22 ftpd_banner=(none - default vsftpd banner is displayed)
23 banner_file=(none) Set to some filename
22 ftpd_banner=(none - default vsftpd banner is displayed)
This string option allows you to override the greeting banner
displayed by vsftpd when a connection first comes in.
23 banner_file=(none) Set to some filename
This option is the name of a file containing text to display
when someone connects to the server. If set, it overrides the
banner string provided by the 'ftpd_banner' option.
-----------------------------
E. Fine-Tuning Access Control
-----------------------------
24 userlist_enable=NO
25 userlist_file=/etc/vsftp.user_list
26 userlist_deny=YES
27 user_config_dir=(none) or somedir/
24 userlist_enable=NO
If YES, all users listed in the file pointed to by the 'userlist_file='
directive will be DENIED ACCESS even before they are asked for a
password.
25 userlist_file=/etc/vsftpd.user_list
Filename examined only when 'userlist_enable=YES'.
26 userlist_deny=YES
If set to NO, then exactly the reverse of the above takes place.
All users listed in the above file will ONLY be allowed access.
ALL OTHERS WILL BE DENIED ACCESS.
This option is examined only if 'userlist_enable' is YES.
27 user_config_dir=(none)
An example will make this very powerful directive clear. It
basically allows a user to have his/her very own
/etc/vsftpd/vsftpd.conf file.
That is, a per-user basis special config file.
[Reminds one of the 'config file=smb.conf.%m' directive in Samba]
Let do this :
1. Create a subdir :
# mkdir /etc/vadapav/
2. Create your very own config file with your per-foo directives
for foo :
# vi /etc/vadapav/swarup
3. Put the following directive in /etc/vsftpd/vsftpd.conf
user_config_dir=/etc/vadapav/
4. Restart vsftpd.
5. Now log in as user 'swarup'.
vsftpd will load and use the settings in
/etc/vadapav/foo
for the duration of the session.
The format of this 'foo' file is in line with the normal
/etc/vsftpd/vsftpd.conf file.
Similar to 'config file' directive in Samba
----------
F. Logging
----------
28 xferlog_enable=NO
29 xferlog_file=/var/log/vsftpd.log
30 xferlog_std_format=NO
31 log_ftp_protocol=NO
28 xferlog_enable=NO <---- No logging done if this is NO --->
If you leave this as NO - the default - then NO logging will take place.
If Yes, all uploads/downloads will be placed in /var/log/vsftpd.log.
You can override this using 'xferlog_file=filename"
Note : Default: NO (but the sample config file by RedHat enables it)
29 xferlog_file=/var/log/vsftpd.log
This option is the name of the file to which we write the transfer log.
The transfer log is only written if the above option 'xferlog_enable=Yes'.
30 xferlog_std_format=NO
Def: No Generates logs in vsftpd's own format
If Yes, generates logs in xferlog log format. You can use Apache/SQUID
log analyzers then
31 log_ftp_protocol=NO
If Yes, all FTP requests/responses are logged
BUT
IFF 'xferlog_std_format=No
Useful for debugging.
----------------
G. Uploads/Downloads
--------------------
32 anon_upload_enable=no
33 ascii_upload_enable=no
34 ascii_download_enable=no
32 anon_upload_enable=no
Allow anon users to upload files
write_enable has to be enabled
The anon upload dir has to be writeable for user ftp
33 ascii_upload_enable=no
34 ascii_download_enable=no
Allow users to upload files
write_enable has to be enabled
----------------
G. Miscellaneous
----------------
35 pam_service_name=vsftpd
35 pam_service_name=vsftpd
This string is the name of the PAM service vsftpd will use.
Note : The manual specifies 'ftp' which is wrong - It is 'vsftpd'
****************************

Samba Server

2008-03-29T04:56:00.000+05:30

[global]
SERVER CONFIG OPTIONS

workgroup = VASHILUG
netbios name = MARUTI
encrypt passwords = yes
smb passwd file = /etc/samba/smbpasswd
server string = Samba %v at %h (.bom.labs.net)
time server = yes
keepalive =300
deadtime =0
#config file = /etc/samba/smb.conf.%m
#include = /etc/samba/smb.conf.%m

NETWORKING CONFIG
hosts allow = 192.168.0.
##allow hosts
hosts deny = ALL EXCEPT 192.168.0.10 192.168.0.20 127.0.0.1
##deny hosts
interfaces = 192.168.0.20/24 127.0.0.1
bind interfaces only = yes

LOGGING CONFIG OPTIONS
log file = /var/log/samba.log.%m
log level =1
##debug level
max log size = 100
debug timestamp =yes
## timestamp logs
debug pid =yes
debug uid =yes
syslog =1
syslog only =no

HOME SHARES SECTION
[homes]
comment = Home Directories
browseable =no
##browsable
writable = yes
## writeable = yes
## write ok = yes
DISK SHARES SECTION - BASIC DISK SHARES CONFIG OPTIONS
[vadapav]
available = no
browseable =yes
writeable = yes
read only = no
[accts]
path = /accounts
## directory
copy = vadapav
available = yes
comment = Accounts Dept
[fin]
path = /finance
copy = vadapav
available = yes
comment = Finance Dept
[hr]
path = /hrd
copy = vadapav
available = yes
comment = HR Dept
[cdrom]
path=/mnt/cdrom
root preexec = mount /dev/cdrom /mnt/cdrom
root postexec = umount /mnt/cdrom
SHARE LEVEL ACCESS CONTROL AND PERMISSIONS
[funny]
#path=/jokes
#writeable=
#hosts allow =
#hosts deny =
#valid users =
#invalid users =
#admin users =
#read list =
#write list =
#max connections =0
##map to guest =Never This is Global but is default anyway.
#guest account =nobody
#guest ok =no
##public
#guest only =no
##only guest=no
SERVER CONFIG OPTIONS [10]
# 1 workgroup = VASHILUG
# 2 netbios name = MARUTI
# 3 encrypt passwords = yes
# 4 smb passwd file = /etc/samba/smbpasswd
# 5 server string = Samba %v at %h (.bom.labs.net) (%m)
# 6 time server = yes
# 7 keepalive=300
# 8 deadtime=0
# 9 config file = /etc/samba/smb.conf.%m
# 10 include = /etc/samba/smb.conf.%m
NETWORKING CONFIG OPTIONS [7]
# 12 hosts allow = 192.168.0.
# 13 #allow hosts
# 14 hosts deny = ALL EXCEPT 192.168.0.10 192.168.0.20 127.0.0.1
# 15 #deny hosts
# 16 interfaces = 192.168.0.10/24 127.0.0.1
# 17 bind interfaces only = yes

VI - LOGGING CONFIG OPTIONS [10]
# 18 log file = /var/log/samba.log.%m
# 19 log level =1
# 20 ##debug level
# 21 max log size = 100
# 22 debug timestamp =yes
# 23 ## timestamp logs
# 24 debug pid =yes
# 25 debug uid =yes
# 26 syslog =1
# 27 syslog only =no
BASIC DISK SHARES CONFIG OPTIONS [11]
# 28 path = /accounts
# 29 #directory
# 30 comment = Home Directories
# 31 read only = no
# 32 writeable = yes
# 33 #writable
# 34 #write ok
# 35 copy = template-1
# 36 available = yes
# 37 root preexec = mount /dev/cdrom /mnt/cdrom
# 38 root postexec = umount /mnt/cdrom

SHARE LEVEL ACCESS CONTROL AND PERMISSIONS [12]
# hosts allow
# hosts deny
# 39 valid users =
# 40 invalid users =
# 41 admin users =
# 42 read list =
# 43 write list =
# 44 max connections =0
# 45 # map to guest =Never This is Global but is default anyway.
# 46 guest account =nobody
# 47 guest ok =no
# 48 #public
# 49 guest only =no
# 50 #only guest=no
## "Note that guest ok parameter must be set to ok for this to work"
########################## 50 Directives #######################

LILO boot error codes

2008-03-29T04:52:00.000+05:30

During the Boot Process,When LILO loads itself, it displays the word LILO.
Each letter is printed before or after performing some specific action. If LILO
fails at some point, the letters printed so far can be used to identify the problem.
Output Problem
(nothing) No part of LILO has been loaded. LILO either isn't installed or the
partition on which its boot sector is located isn't active.

L The first stage boot loader has been loaded and started, but it can't load the second stage boot loader. The two-digit error codes indicate the type of problem. This condition usually indicates a media failure or a geometry mismatch (e.g. bad disk parameters).
LI The first stage boot loader was able to load the second stage boot loader, but has failed to execute it. This can either be caused by a geometry mismatch or by moving /boot/boot.b without running the map installer.
LIL The second stage boot loader has been started, but it can't load the descriptor table from the map file. This is typically caused by a media failure or by a geometry mismatch.
LIL? The second stage boot loader has been loaded at an incorrect address.
This is typically caused by a subtle geometry mismatch or by moving /boot/boot.b without running the map installer.
LIL- The descriptor table is corrupt. This can either be caused by a
geometry mismatch or by moving /boot/map without running the map
installer.
LILO All parts of LILO have been successfully loaded.

If the BIOS signals an error when LILO is trying to load a boot image,
the respective error code is displayed. These codes range from 0x00
through 0xbb.
See man lilo for explanation of these codes

User Administration

2008-03-29T04:48:00.000+05:30

1. Create a user :
# adduser raj
# grep raj /etc/passwd /etc/shadow
/etc/passwd:raj:x:513:513::/home/raj:/bin/bash
No point looking at /etc/passwd since nothing
changes there
/etc/shadow:raj:!!:13034:0:99999:7:::

2. chage -d 0 raj
The 3rd field of /etc/shadow is :
Days since Jan 1, 1970 that password was last changed
We make it 0, which means the passwd was last changed on
Jan 1, 1970 and hence has expired, so it is promptly locked
with this : --> !!
# grep raj /etc/shadow
/etc/shadow:raj:!!:0:0:99999:7:::

3. Test the password status :
# passwd -S raj
Password locked.
raj cannot log in, yet

4. Unlock the password, forcefully
# passwd -uf raj
Unlocking password for user raj.
passwd: Success.
This changes the !! to a blank which means the password is now unlocked

Faqs

2008-03-29T04:37:00.000+05:30

1. When u press Alt+F2 and login in, which files are executed ?
Ans.
/etc/issue
$HOME/.hushlogin iff not exists
* mingetty checks for .hushlogin in $HOME
If $HOME/.hushlogin DOES NOT EXISTS then it does the foll :
- execs lastlog* -u $USER using /var/log/lastlog
- execs /etc/motd
- checks users mail, if any
/etc/profile
/etc/inputrc
/etc/termcap
/etc/profile.d/*.sh [13 shell scripts executed]
/etc/bashrc
$HOME/.bash_profile
Misc: $HOME/.bash_history
$HOME/.bash_logout
$HOME/.bashrc

2. LILO to GRUB : Change your bootloader
Ans.
# grub-install /dev/hda
Config file : /etc/lilo.conf
If you re-install Win$, you must boot from a boot floppy and do a 'lilo -v'

3. GRUB to LILO : Change your bootloader
Ans.
a. cp /etc/lilo.conf.anaconda /etc/lilo.conf
b. lilo -v [-t]
Config file : /boot/grub/grub.conf
: /etc/grub.conf [symlink to above]
If you re-install Win$, no effect on Linux since MBR does not contain
code except for a small loader which loads the main grub in 2 stages :
/usr/share/grub/i386-redhat/stage1 512 bytes
/usr/share/grub/i386-redhat/stage2 104K

4. To put a password in GRUB :
Ans.
password --md5 $1$JfobY0$HevBan5wGn.C/eteLH/jT1
# grub-md5-crypt
>> vashi123
>> vashi123
Now copy / paste this readable but scrambled password in grub.conf
It is the md5 encrypted equivalent of 'vashi123'

5. To uninstall Linux :
Ans.
# fdisk /dev/hda and delete all Linux partitions
# lilo -u Uninstall lilo
or
# Boot with Win$ startup disk
A:> fdisk /mbr
A:> Now reboot and you will have Win$ only

6. LILO : To create a boot floppy if your boot loader is LILO
Ans.
# uname -r
2.4.20-8
# mkbootdisk 2.4.20-8
Obviously the floppy should not be mounted !
Shorter version :
# mkbootdisk `uname -r`

7. GRUB : To create a boot floppy if your boot loader is GRUB
Ans.
Get a working floppy !
1. # mke2fs /dev/fd0
2. # mount /mnt/floppy
3. # cd /mnt/floppy
4. # mkdir -p boot/grub
5. # cd /boot/grub/
6. # cp stage1 /mnt/floppy/boot/grub/
7. # cp stage2 /mnt/floppy/boot/grub/
8. # cp grub.conf /mnt/floppy/boot/grub/
9. # grub --batch <<> root (fd0)
> setup (fd0)
> quit
> EOT
The moment you press Enter, a Boot grub floppy is created.
Boot with it to test it!

8. To Change the splash screen in grub :
Ans.
Using ImageMagick-5.4.7-10 software
* /usr/bin/identify -v bear.jpg
* /usr/bin/convert
* /usr/bin/animate
* /usr/bin/composite
* /usr/bin/conjure
* /usr/bin/display
* /usr/bin/import
* /usr/bin/mogrify
* /usr/bin/montage
Consider bear.jpg
1. convert -depth 8 -colors 14 -resize 640x480 bear.jpg bear.xpm
Has to be max 14 colors
Has to be 640x480 resolution
Has to be xpm format [X Windows System pixmap (color)]
See man ImageMagic
2. Now gzip the bear.xpm
# gzip bear.xpm -------> bear.xpm.gz
3. Copy it to /boot/grub/
# cp bear.xpm.gz /boot/grub/
4. Configure /boot/grub/grub.conf
splashimage=(hd0,11)/grub/bear.xpm.gz
5. reboot

Linux Boot Process

2008-03-29T04:10:00.000+05:30

======================
The Linux Boot Process
=======================
Part I
======
01. Power On
02. ROM - BIOS loaded into RAM auto by the boot code at the start of ROM [bootstrap]
Purpose of BIOS : Load the OS - any/some OS
03. POST - HW Inventory - Int 19H
04. CMOS - [Current bootable devices]
05. BIOS loads/starts [boots] whatever it finds in the first Sector [Sect 1] of HDD into RAM i.e. [cylinder 0, head 0, sector 1 of a hard disk]
- This first Sector is aka the MBR [on Windows-based systems]
Note: There is only one MBR per HDD, obviously !!
** During installation, when the Linux kernel is being installed, its exact size and location [CHS] on the HDD is stored in /boot/map in the /boot partition. By that we mean, the exact CHS no on the HDD of where it installed /boot/vmlinuz-2.4.20-8 Tis is the address of the Linux kernel on the HDD
06. BIOS then loads/executes [boots/runs] this first stage BL - which is
LILO [i.e /boot/boot.b] which is in the BS of the MBR and put there at install time AND NOT
/boot/boot.b from the /boot partition] and Transfers control to the it
Tis is called the 1st stage of bootloading
07. This first-stage BL has but one purpose in life : To load the 2nd stage BL.
08. Makes BIOS calls - Int 13 fn 8 --> "Get drive parameters" and - Int 13 fn 2 --> "Read sectors from drive" - which returns the disk geometry in 24-bit address format upto the
1024 cylinder limit.
Since you have enabled the LBA mode, 64-bit address format is used and
instead of the CHS way, BIOS reads the disk by numbering the sectors
from 1,2,3... etc
The requested info from this BIOS call is stored by the BIOS in the first
page of memory ( < 4k )
09. The first-stage loader then requests the BIOS for an Int 13 fn 2, gives it
the CHS # of the /boot/boot.map file
Now the BL speaks CHS and the BIOS speaks 24-bit addresses.
It is LBA which allows the BIOS to act as a speaker of both, CHS and
Linear Sector Addressing - using 64-bit addresses - Translator
10. BIOS loads this 2nd stage BL - from the CHS no supplied by the 1st stage
11. The 2nd-stage BL now loads the contents of the balance of the BS into the RAM
And what is that ?
/etc/lilo.conf
Not the file lilo.conf in dir /etc
but the contents of the BS - which is the same file - but in binary form
and was put there by the installer
12. The 2nd stage BL now examines this binary-format-BS which contains
"lilo.conf" and follows the instructions given there !!
It requests BIOS for an Int 13h and gives it the CHS # of the
/boot/message file which BIOS promptly fetches and the 2nd stage BL
executes - This is the splash screen
13. The 2nd stage BL then executes the instructions it got from the BS
after the 1st stage had loaded it, and executes those instructions
/etc/lilo.conf
==============
prompt <--------- Display labels
timeout=50 <--------- In decisecs
default=linux <--------- What to do after timeout
boot=/dev/hda <--------- Where to install the 1st stage BL
Here, in the BS of the MBR of Primary
Master HDD
map=/boot/map <--------- Location of map file which contains size and location of
ll=/boot/boot.b <--------- Boot sector image/file for MBR or
1st stage LILO to be installed in BS of the MBR

Message=/boot/message<-------- Splash Screen
lba32
image=/boot/vmlinuz-2.4.20-8 <------ Big Boss - The Linux kernel
label=linux <------ An unscary word for the kernel
initrd=/boot/initrd-2.4.20-8.img <--- Image of support modules
read-only
append="hdc=ide-scsi root=LABEL=/" <--- Parameters LILO will pass to
init before it starts
other=/dev/hda1 Another OS
optional
label=DOS
14. After timing out AND IF the default image is the Linux kernel, and IT IS
in the case above, the 2nd stage boot loader - /boot/boot.b -
loads the Linux kernel in RAM
15. Now what is a BootLoader ?
BootLoaders load "kernel images" into mem and hand control of the CPU
over to the newly loaded kernel [/boot/vmlinuz-2.4.20.8]
To make your new kernel work, you must tell the BL about the kernel
Nearly all BLs use and understand CHS numbers.
[BIOS can now understand CHS nos too, bcos of LBA]
16. Now the 2nd-stage wishes to load the kernel, as instructed by the
binary-format lilo.conf after timing out or whatever the user chooses
But there's a problem !!
The Linux kernel is on the HDD !!
In /boot to be precise !!
And the 2nd-stage LILO BL has no idea how to access the HDD
Why ?
The HDD drivers - /dev/hda... are in /dev on the HDD
and to access it the 2nd stage BL requires the file /dev/hda
which once again is itself on the HDD !!
Hen-and-Egg situation !!
17. The 2nd stage BL has no choice but to use the BIOS as its HDD device
driver to access the HDD at boot time.
After all that's where the kernel is [/boot/vmlinuz-2.4.20-8] !!
18. The 2nd stage BL once again requests BIOS - Int 13h fn 2 - and hands it
the CHS # of the /boot/map file, which BIOS loads
19. The 2nd stage loader now examines this /boot/map and extracts the
CHS # inside it - which was put there by the BL installer - anaconda -
and /sbin/lilo - and requests BIOS once more to load whatever is there
in that CHS # address
17. BIOS then loads the Linux kernel /boot/vmlinuz-2.4.20-8 into the RAM
and hands it to the 2nd stage BL
18. The 2nd stage BL uncomptars this file and fills the memory with kernel
mem structures which can be seen in the /proc virtual file system
The Engine Of The Car Is Ready And Running !!
The Linux OS is ready and running !! Kernel Land is ready !!
BUT WE FORGOT TO MAKE THE CAR PROPER ITSELF !! No Userland !!
How do we drive this car if there is no seat, steering wheel. Nothing.
Just an engine !!
=============================
Part II /sbin/init takes over
=============================
19. Now that the Linux Kernel is in the RAM, and is much more powerful
and immensely larger than the 2nd stage BL, it takes over total control
from it and makes yet another BIOS call Int 13h fn 2 for the
file /boot/initrd-2.4.20-8.img
20. It uncompresses it in RAM and creates a mini-FS as can be seen if
we look inside the image
# mount -o loop /boot/initrd-2.4.20-8.img /win
# cd /win
# ls
21. Note that the HDD driver is available now; Hence the kernel executes the
"linuxrc" script which mounts the "root" partition in R/O mode and
loads the /sbin/init* program into RAM
The Userland Part of Linux is now started by the /sbin/init daemon
22. /sbin/init* examines /etc/inittab

/etc/inittab
============
id:3:initdefault:
# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6
ls:S:wait:/etc/rc.d/rc S
~:S:respawn:/sbin/sulogin
# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"
# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Cancelled"
# Run gettys in standard runlevels
1:2345:respawn:/sbin/mingetty --noclear tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6
11:2345:respawn:/sbin/mingetty tty11
# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon
=======================================================================
23. a. Executes /etc/rc.d/rc.sysinit in a subshell of inittab shell
rc.sysinit
==========
calls /etc/sysconfig/network in same env [shell]
sets the hostname and checks whether NW is yes or no
ie checks for existence for NW card
calls /etc/init.d/functions script in the same env [i.e rc.sysinit]
sets global umask and path and defines 17 shells fns
within the rc.sysinit shell env
# cat functions grep sortnl
/etc/init.d/functions
=====================
1 action() {
2 checkpid() {
3 confirm() {
4 daemon() {
5 echo_failure() {
6 echo_passed() {
7 echo_success() {
8 echo_warning() {
9 failure() {
10 killproc() {
11 passed() {
12 pidfileofproc() {
13 pidofproc() {
14 status() {
15 strstr() {
16 success() {
17 warning() {
24. Displays the "Welcome to Red Hat..." from /etc/redhat-release
25. Runs dmesg* which creates /var/log/dmesg and
display its contents on the screen
26. Mounts all local filesystems from /etc/fstab and updates /etc/mtab
Mount simply displays the contents of /etc/mtab
Now /etc/rc.d/rc.sysinit script ends !!
Summary: rc.sysinit : 1. sets the global umask, global PATH,
sets up the NW'ing subsystem
Basically starts the System Daemons
b. Back to inittab
27. rc is executed in a new subshell of /etc/inittab
starts and stops the Application NWing daemons
runs /etc/rc.d/rc.local which is a symlink in all RLs and is the
only script which is run regardless of the RL and is
S99local in all /etc/rc.d/rc?.d/...] and points to
/etc/rc.d/rc.local
goes back to inittab
Summary: rc : starts the 3rd party NW services [User NW services]
Basically starts the Application Daemons
i.e all the * in ntsysv

28. c. shutdown CAD magic keys are set - You can hack it if you wish
29. d. poweroff and poweron considerations [start upsd* daemon "ups" initscript
which reads /etc/sysconfig/ups which you will configure for your UPS model

30 e. 6 mingettys are spawned with dev drvs ; all enter sleep state
- EXCEPT one
31.f. Iff Runlevel 5, then script "prefdm" is executed..
we will do this in class of X
NOW INITTAB IS OVER
========
Part III
========
Note:
What is a getty? [RHL has agetty* [Alternate Linux getty]
mgetty* [Modem getty]
mingetty* [Minimal getty]
A getty is is a program that opens a tty port, prompts for a login name,
and runs the /bin/login command. It is normally invoked by init.
32 g. The first mingetty [awake one] loads dev drv /dev/tty1
* execs /etc/issue [Magic cookies Allowed: man uname / man mingetty]
33. * mingetty then executes the /bin/login process
and is then put to sleep state by init
34. * /bin/login displays login. You login!
35. * /bin/login execs /usr/bin/passwd which challenges for passwd from
user which then does authentication and authorization using PAM
36. * init then puts the login daemon into sleep state, wakes up
mingetty which now takes over
37. * mingetty checks for .hushlogin in $HOME/$USER
38. If $HOME/.hushlogin DOES NOT EXISTS
then it does the foll :
39. - execs lastlog* -u $USER using /var/log/lastlog
- execs /etc/motd
- checks users mail, if any, in /var/spool/mail/$USER
40. * init then wakes /bin/login process
41. * mingetty then goes into zombie state and is killed by init
42. * login then loads /bin/bash as a monitored child process
43. * /bin/bash takes over and login goes into sleep state
44. * Execs /etc/profile Sets system-wide ENV variables Global Profile
exec /etc/inputrc Sets keyb mappings [See stty -a]
Show Terminal characteristics
eg # stty intr ^g --> Keyboard mapping
# stty -echo --> Terminal Characteristics
eg /etc/inputrc See man bash - /bell-style
============
set bell-style [audible] [none] [visible]
set disable-completion [off] on
To do it per-user:
=================
setterm -blength 0 [in per user .bash_profile]
or
xset b off [in GUI]
[bash -r, --noediting as args in /etc/passwd
7th field]
45. exec /etc/termcap Sets the terminal capabilities
46. exec /etc/profile.d/*.sh [13 shell scripts are executed]
1. colorls.sh
2. glib2.sh
3. gnome-ssh-askpass.sh
4. krb5.sh
5. lam.sh
6. lang.sh
7. less.sh
8. mc.sh
9. pvm.sh
10. qt.sh
11. vim.sh
12. which-2.sh
13. xpvm.sh
47. i. * Execs /etc/bashrc - Sets system-wide [Global] aliases/shell functions
sets the users and root's final global umask
48. * execs user's personal .bash_profile
49. execs .bashrc Run after every command you execute
50. .bash_history
51. .bash_logout
52. Finally, at last, the comforting shell login prompt bash
inviting you type a cmd and explore the party [FS] !!!!

Comparison between DOS and Linux Commands

2008-03-29T03:13:00.000+05:30

DOS command // Linux Commands
1. dir // ls
2. dir/w // ls -l aka ll
3. dir/p // ls -lmore ; ls -lless
4. dir/s // ls -R recursive
5. cls // clear
6. type // cat
7. cd // pwd
8. md // mkdir
9. rd // rmdir If empty
10. del // rm
11. move/ren // mv
12. copy // cp
13. xcopy // cp -a
14. deltree // rm -fr