Awk and other text processing tips

awk is great for working with data that is in several columns. 

How to sum the third column?

e.g., calculate total tps across all physical disks from iostat -d output:

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 3.73 30.38 67.80 4864950 10857636
sdb 3.82 30.39 71.74 4866793 11488576
sdc 0.00 0.05 0.00 7208 8

iostat -d |egrep "sd.\ " | awk 'BEGIN {x=0} {x+=$2} END {print x}'

or, less elegantly,

iostat -d| egrep "sd.\ " | awk 'BEGIN {ORS=""}; {print $2"+"}' | ( cat; echo 0)|bc

How to grab just certain columns?

How do I use awk to print the first column, and then the third through the end, for example to grab just the fields I want from an apache log file?

awk '{ print $1" " substr($0, index($0,$6)) }' /var/log/httpd/access_log*

gives us something like

10.95.10.20 "POST /license/associateproduct.php HTTP/1.1" 200 8 "-" "Java/1.6.0_17"
10.95.14.248 "POST /license/authorize.php HTTP/1.1" 200 84 "-" "PycURL/7.19.5"
 

NDMP Backups with EMC Networker - Quick Start

Table of Contents

Background

Networker Server Setup

Install Base OS and prerequisites

Install and configure Networker App

Configure Licensing for NDMP Backups

Configure backup location, pool, and media (disk based backups are fast for testing)

Client/Management Station Setup

Install Networker Client and Management Center

Configure the Management Console and Networker Server

Configuring for NetApp FAS

Prep the FAS

Prep Networker

Perform the Backup

Configuring for EMC Celerra

Prep Networker

Prep the Celerra

Perform the Backup

Background

Why NetApp and/or EMC has not made such a short and simple guide (that I could find), why a humble sys-admin like myself has had to go through the trial and error process to figure this all out, is only a reflection on the fact that EMC (especially EMC, but to some extent NetApp also,) has become so large as to no longer realize that customers are there. Someone checks "Product Manual" off a release check-list, and calls it good. But none of those people evaluate whether the manual is any good, instead preferring to up-sell to $5000 of videos, or on-site consultants. Nobody thinks that, "Hey, this customer ordered the NDMP option; We should send them a quick-start guide to help them get started with some of the big partners".

This quick guide does provide a quick start guide to performing a disk-based backup via NDMP of your NetApp Filer or EMC Celerra NAS with EMC Networker 7.4. This quick guide does not cover the ins and outs of each platform, data archival and disaster preparedness best practices, etc. This guide was set up using my environment at a company where I know longer work; I cannot provide additional support.

If you are evaluating EMC Networker, ask your sales rep for evaluation license codes for the NDMP and disk backup features; otherwise, you'll need to buy them.

The license codes are all temporary, 30 days.

Networker Server Setup

Install Base OS and prerequisites

Install CentOS 5 i386. Install the X window system (no gnome or KDE is required), as some X libraries are a prerequisite for the networker client (and perhaps other components

Install libgcj and java-1.4.2-gcj-compat for the graphical java components.

service iptables stop

chkconfig iptables off

Disable selinux (set to disabled in /etc/sysconfig/selinux).

Install and configure Networker App

Extract the networker server Linux files as follows:

tar xzf nw74_linux_x86_64.tar.gz

Install the networker server linux component rpms as follows:

rpm -ivh lgtoclnt-7.4-1.i686.rpm lgtolicm-7.4-1.i686.rpm lgtoman-7.4-1.i686.rpm lgtoserv-7.4-1.i686.rpm lgtonode-7.4-1.i686.rpm lgtonmc-3.4-1.i686.rpm

If using DHCP, add DHCP_HOSTNAME=<hostname> to /etc/sysconfig/network-scripts/ifcfg-eth0 . This will cause the DHCP client to also register the hostname in DNS.

Add an entry for the eth0 ip address and hostname to /etc/hosts.

Run these commands to configure and start networker services

/etc/init.d/networker stop

/etc/init.d/gst stop

/opt/lgtonmc/bin/nmc_config

[root@networker ~]# /opt/lgtonmc/bin/nmc_config

NOTE

====

Install has detected the configuration file of a previous lgtonmc

package. Install will attempt to read the configuration parameters

in this file and present them as default values where appropriate.

Please modify any value that is incorrect or needs to be changed.

The Command Line Reports feature of NetWorker Management

Console requires a Java Runtime Environment (JRE) be

installed on this machine. The JRE version should be

1.4.2 or higher, up to (but not including) 1.6.

Is there a supported JRE already installed on this machine [y]? y

Please specify the directory where JRE is installed [/usr/lib/jvm/jre]? /etc/alternatives/jre_1.4.2/

What port should the web server use [9000]?

What port should the GST server use [9001]?

What directory should be used for the LGTOnmc database [/opt/lgtonmc/lgto_gstdb]?

/opt/lgtonmc/lgto_gstdb/lgto_gst.db already exists, do you want to retain this database [y]? n

/opt/lgtonmc/lgto_gstdb/lgto_gst.db already exists, is it okay to remove it [n]? y

Where are the NetWorker binaries installed [/usr/sbin]?

Start daemons at end of configuration [n]?

Creating installation log in /opt/lgtonmc/logs/install.log.

Performing initialization. Please wait...

Installation successful.

/etc/init.d/networker start

/etc/init.d/gst start

chkconfig networker on

chkconfig gst on

Run nsradmin. Type “visual” at the “nsradmin>” prompt to go into visual edit mode. Select type: “NSR”, then you should see your networker server. Select “Edit”, then key down to the “administrator” field. Add ,“*@*” – WARNING: this is only for testing, and will enable anybody on any machine to affect your backups. DON’T FORGET TO CHANGE IT.

Configure Licensing for NDMP Backups

(See client setup for instructions on accessing the Networker Management Console.)

In the Networker Management Console, click Setup LicensingNew, and enter an enabler code for “NetWorker NDMP Client Connection for NetWorker” or “NetWorker NDMP Client Connection – Tier 1”, Networker Server, Storage, DiskBackup, NDMP for NetApp (use your tier in each case) as needed.

Configure backup location, pool, and media (disk based backups are fast for testing)

(See client setup for instructions on accessing the Networker Management Console.)

In the NMC, double-click the server name of your Networker Server.

Go to Devices  Nodename  Storage Nodes  Nodename; right-click and select “Properties”. Change “Node Type” to NDMP.

Add storage location (device) for disk based backups on the Networker Storage node:

Go to Devices  Nodename  Devices, Right-click, and select “New”. For “Name:”, enter the backup path, and for “Media Type” enter “file”. Click “OK”. Then double-click the newly-created device again to pull up more configuration options.

Create a media pool for the disk-based backup:

Go to Media <Server Name>Media PoolsNew

Label the media for the device. Go to Devices<Server Name>Devices, right-click your disk backup device, and select “Label”. Change the pool to “disk backup pool” (or whatever your pool was). Note: if you don’t label it to the right pool, you’ll get a perpetual notice in /nsr/logs/messages: “NetWorker media: (waiting) Waiting for 1 writable volumes to backup pool 'disk based backup' disk(s) on <servername>”.

Ensure that a volume exists for the media pool. You may have to create this manually

Go to Media <Server Name> Volumes

Client/Management Station Setup

Your workstation will be the management station.

Install Networker Client and Management Center

This table describes the browser, OS, and JRE requirements:

Platform	Browser
AIX	Mozilla 1.7
HP/UX	Mozilla 1.6
Red Hat Enterprise Linux Server 3, 4 and 5 SuSE Linux Enterprise Server 8, 9 and 10	Mozilla 1.7
Solaris 8 or 9	Netscape 7 Mozilla 1.7
Solaris 10	Mozilla 1.7
Windows2003 Enterprise or Datacenter Edition	Internet Explorer 6.x
Windows XP Windows2000 Windows2003 32-bit	Internet Explorer 5.5 or 6.x
Windows Vista	Internet Explorer 7.0

Environment	Version
Java Runtime Environment	1.4.2 or greater (but not 1.6) for all, except, 1.5.0_11 or greater (but not 1.6) for Windows Vista

You may need to disable the Java 6 runtimes on your workstation from the java control panel (access it in the system control panel.) Also within the Java control panel, you may have to set your temporary file size to 300MB.

Map a drive to \a80IT-Software . Then browse to that drive AppsEMCNetworkerNetworker_7.4networker_7.4-win . Run setup.exe . Select “client” and “networker management console”.

Configure the Management Console and Networker Server

Start up the Networker console by opening a web browser on the management workstation to http://localhost:9000/ , click “start”, and use the credentials administrator:administrator .

Note: The Java app that opens will not receive an icon in the task bar.

Accept the EULA. Change the password as desired. Configure the NLM, Database Backup Server, and Networker server for your networker server FQDN. I receive an error about permissions on the database, but it seems to work okay.

In the Networker Management Console, on the Enterprise tab, your networker server should show up. Click on it, and then double-click the application, “Networker”. This is where you’ll configure

Configuring for NetApp FAS

Prep the FAS

First, ensure that manual entries exist in the hosts files for host name and FQDN for the networker server (e.g., networker), storage node, and NDMP server (e.g., fas270c), each of those nodes. (A correct DNS configuration is not necessarily adequate, and the application’s reverse lookups may fail).

Prep Networker

Configure the client Fas270c:

Go to Configuration <Server Name>ClientsNew

Perform the Backup

Run the command on the legato server:

(this example will perform a full NDMP dump of a volume, a qtree, or a subvolume or sub-qtree)

[root@vm-legato ~]# nsrndmp_save -T dump -M -c fas270c.<fqdn> -g Default /vol/vol0/auto-D

Configuring for EMC Celerra

Prep Networker

Configure the client for the Celerra Data mover in the Networker UI:

Set tape buffer size. (Ignore this step in 5.5.30. default of 128KB; networker disk backup requires 32KB; range is 64-1168)

Prep the Celerra

Log in to the Celerra front-end as nasadmin.

$ server_param server_2 -facility NDMP -modify bufsz -value 32

[nasadmin@ns350cs nasadmin]$ server_param server_2 -facility NDMP -list

server_2 :

param_name facility default current configured

maxProtocolVersion NDMP 4 4

convDialect NDMP '8859-1' '8859-1'

scsiReserve NDMP 1 1

dialect NDMP '' ''

includeCkptFs NDMP 1 1

md5 NDMP 0 0

forceRecursiveForNonDAR NDMP 0 0

snapTimeout NDMP 5 5

bufsz NDMP 128 128

snapsure NDMP 0 0

v4OldTapeCompatible NDMP 1 1

create ndmp user on data mover

[root@ns350cs root]# /nas/sbin/server_user server_2 -add -password ndmp

Creating new user ndmp

User ID: 1000

Group ID: 1000

Comment: NDMP user

Home directory: /home/ndmp

Shell:

Changing password for user ndmp

New passwd: ndmppass

Retype new passwd: ndmppass

Perform the Backup

[root@vm-legato ~] nsrndmp_save -T dump -M -c 192.168.123.156 -g Default /siq_daily.2/foo

PAM notes

Much of these notes are taken from the Linux-PAM man page, from http://www.kernel.org/pub/linux/libs/pam/Linux-PAM-html/Linux-PAM_SAG.html, and from http://www.hccfl.edu/pollock/AUnix2/PAM-Help.htm .

Linux PAM (Pluggable Authentication Modules) "is a system of libraries that handle the authentication tasks of applications (services) on the system."

Configuration for individual applications/services may reside in /etc/pam.d . If that directory does not exist, then PAM will look for the single config file /etc/pam.conf . The configuration file(s) define the connection between applications (services) and the pluggable authentication modules (PAMs) that perform the actual authentication tasks.

PAM policy/config file syntax
Each line (rule) in a policy file has 4 parts:

1. context (service type) - what aspect of the user's request for a restricted service does this line affect?
1. auth -- (authentication) authenticate a user and set up user credentials. Authentication means that the user proves his identity; typically, this involves entering a password, but it may include a hardware based authentication scheme (e.g., smart card). The setting up of user credentials may include setting up group memberships or other privileges.
2. account -- (authorization?) -- provide account verification types of service: e.g., has the user's password expired? is this user permitted access to the requested service at this time? Are sufficient system resources available? is this account allowed on the console?
3. password -- update authentication tokens, for example, "please enter a new password". Typically, there is one module for each challenge/response based auth type.
   
4. session -- session setup and cleanup; covers things that should be done prior to a service being given, and after it is withdrawn. For example, leaving audit trails, mounting the user's home dir, or unmounting it after logoff.
2. control - tells PAM how to handle a "fail" result from a module's authentication task. There are two types of syntax for this control field: the simple one has a single simple keyword; the more complicated one involves a square-bracketed selection of value=action pairs.
1. For the historical/simple control field, valid values are
   
1. required -- "if fail, then ultimately fail, but first finish the remaining stacked modules." failure of such a PAM will ultimately lead to the PAM-API returning a failure after the remaining stacked modules (for this service and type) have been invoked.
2. requisite -- "if fail, then return to the app now with a failure". like required, but in the case where such a module returns a failure, control is directly returned to the application (without attempting the other stacked modules). The returned value is that associated with the first required or requisite module to fail. This flag may protect, for example, against the possibility of a user getting the opportunity to enter credentials over an unsafe medium; but it may also inform an attacker of valid accounts on a system.
3. sufficient -- "if a prior required module has not failed, then a success here is good enough to return to the app immediately with success". success of such a module is enough to satisfy the authentication requirements of the stack of modules (if a prior required module has failed, then the success of this one is ignored). A failure of this module is not deemed as fatal to satisfying the application that this type has succeeded. If the module succeeds, the PAM framework returns success to the application immediately, without trying other modules
4. optional -- the success of this module is only important if it is the only module in the stack associated with this service+type.
5. include -- include all lines of given type from the configuration file specified as an argument to this control. (On recent RH-based systems, individual application files will tend to include system-auth instead of "other".)
   
6. substack -- include all lines of given type from the configuration file specified as an argument to this control. This differs from include in that evaluation of the done and actions in a substack does not cause skipping the rest of the complete module stack, but only of the substack. Jumps in a substack also can not make evaluation jump out of it, and the whole substack is counted as one module when the jump is done in a parent stack. The reset action will reset the state of a module stack to the state it was in as of beginning of the substack evaluation.
   
3. module path - the PAM module being called
4. module arguments (optional) -- options passed to the PAM module. This is a space separated list of tokens that can be used to modify the specific behavior of the given PAM. See individual module documentation for details of that module's options. For arguments that include spaces, surround that argument with square brackets.
   

In the case where the /etc/pam.conf file is used, an additional service field (e.g. login or su, or other (=default)) appears at the beginning of each line, and describes which service the line applies to.

The lines/modules are run in the order in which they occur in the file. They're passed the module options (if any) and user/request info, and they return a pass/fail (or other) result. The modules are run until an overall pass/fail result is reached, and that result is passed back to the service to which the user has requested access.

Any line in the config file that is not formatted correctly will generally tend to make the authentication process fail.

Errors are written to syslog.

Detail on Control types

1. For the historical/simple control field, valid values are
   
1. required -- failure of such a PAM will ultimately lead to the PAM-API returning a failure after the remaining stacked modules (for this service and type) have been invoked.
2. requisite -- like required, but in the case where such a module returns a failure, control is directly returned to the application (without attempting the other stacked modules). The returned value is that associated with the first required or requisite module to fail. This flag may protect, for example, against the possibility of a user getting the opportunity to enter credentials over an unsafe medium; but it may also inform an attacker of valid accounts on a system.
3. sufficient -- success of such a module is enough to satisfy the authentication requirements of the stack of modules (if a prior required module has failed, then the success of this one is ignored). A failure of this module is not deemed as fatal to satisfying the application that this type has succeeded. If the module succeeds, the PAM framework returns success to the application immediately, without trying other modules
4. optional -- the success of this module is only important if it is the only module in the stack associated with this service+type.
5. include -- include all lines of given type from the configuration file specified as an argument to this control. (On recent RH-based systems, individual application files will tend to include system-auth instead of "other".)
   
6. substack -- include all lines of given type from the configuration file specified as an argument to this control. This differs from include in that evaluation of the done and actions in a substack does not cause skipping the rest of the complete module stack, but only of the substack. Jumps in a substack also can not make evaluation jump out of it, and the whole substack is counted as one module when the jump is done in a parent stack. The reset action will reset the state of a module stack to the state it was in as of beginning of the substack evaluation.
2. The more complicated syntax is as follows:
1. valid control values have the following form: [value1=action1 value2=action2 ...]
2. ...where valueN corresponds to the return code from the function invoked in that module line, and is one of these:
1. success, open_err, symbol_err, service_err, system_err, buf_err, perm_denied, auth_err, cred_insufficient, authinfo_unavail, user_unknown, maxtries, new_authtok_reqd, acct_expired, session_err, cred_unavail, cred_expired, cred_err, no_module_data, conv_err, authtok_err, authtok_recover_err, authtok_lock_busy, authtok_disable_aging, try_again, ignore, abort, authtok_expired, module_unknown, bad_item, conv_again, incomplete, and default.
3. ActionN can be either an unsigned integer, n, signifying an action of "jump over the next n modules in the stack', or it can take one of the following forms:
1. ignore -- when used with a stack of modules, the module's return status will not contribute to the return code the application obtains.
2. bad -- this action indicates that the return code should be thought of as indicative of the module failing. If this module is the first in the stack to fail, its status value will be used for that of the whole stack.
3. die -- equivalent to bad, with the side effect of terminating the module stack and PAM , and immediately returning to the application
4. ok -- this tells PAM that the administrator thinks this return code should contribute directly to the return code of the full stack of modules. IE, if the former state of the stack would lead to a return of PAM_SUCCESS, then the module's return code will override thei value. However, if the former state of the stack holds some value that is indicative of a module's failure, this 'ok' value will not be used to override that data.
5. done -- equivalent to ok, with the side effect of terminating the module stack and PAM immediately and returning to the application.
6. reset -- clear all memory of the state of the module stack and start again with the next stacked module.
4. Each of the four keywords, required, requisite, sufficient, and optional, have an equivalent expression in terms of the [...] syntax as follows:
1. required -- [success=ok new_authtok=ok ignore=ignore default=bad]
2. requisite -- [success=ok new_authtok=ok ignore=ignore default=die]
3. sufficient -- [success=done new_authtok=done default=ignore]
4. optional -- [success=ok new_authtok_reqd=ok default=ignore]

Fix Absolute Symbolic Links from tar or rsync SNAFU

Tar will strip leading slashes from sym links (if you don't use -P), as will rsync in some situations (for example, with chroot=true). This can put you in a world of hurt. Here are some tips for how to fix it.

• This one won't work for a couple of reasons; first, it can't distinguish between intended relative paths and accidental relative paths; also, cut won't work if any files that have a different character length in any fields of the ls output, such as ownership or date (there's a way to merge whitespace that escapes me, not fmt or pr, but something :

• find . -type l -print0 | xargs -0 ls -laQ | grep ' -> \"relative/path/stuff'| cut --complement -d " " -f 1-8 | sed -e 's/ -> \"/\;\"\//g' | awk -F\; '{ print "ln -sf " $2 " " $1 }' > fix_symlinks
• How about this:
• find . -type l -printf 'ln -sf "/%l" "%p"\n' | grep "\"/accidental/relative/path" > fix_symlinks
• check the fix_symlinks for things that you actually don't want to change.
• . ./fix_symlinks

There has got to be a more elegant way to do this. *sigh* if my perl skills were half-decent, it'd probably be a breeze.

Also, note the "symlinks" command in the "symlinks" package.

Pixelated windows graphics fix

I had a new laptop, XP Pro SP2, IE6. Graphics in IE and in the start menu and taskbar were blocky, pixelated.

The problem was, Dell knew the display was hi-res and small, so they set a registry setting for me to automatically dumb-down the graphics, to make them more legible.

This made web pages render incorrectly, and it made the screen icons look bad. Change this key to fix it.

[HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main]
change
"UseHR"=dword:00000000

Who says only Linux has silly things like this, or that Windows is more user-friendly? My grandma would never have found this fix.

PostgreSQL Tips and Tricks


# Log in to a postgreSQL database
psql -U username dbname

# Get list of databases.
SELECT datname FROM pg_database;
# or
\l
# or
psql -l

# Get size of tables, sorted in descending order.
SELECT relname, relpages,reltuples FROM pg_class ORDER BY relpages DESC;

# Show tables in a database.
select * from information_schema.tables where table_schema='public' and table_type='BASE TABLE';
# or
\dt

# Show structure of a table.
\d tablename

psql Commands

General

\c[onnect] [DBNAME|- [USER]]	connect to new database (currently "dfdata")
\cd [DIR]	change the current working directory
\copyright	show PostgreSQL usage and distribution terms
\encoding [ENCODING]	show or set client encoding
\h [NAME]	help on syntax of SQL commands, * for all commands
\q	quit psql
\set [NAME [VALUE]]	set internal variable, or list all if no parameters
\timing	toggle timing of commands (currently off)
\unset NAME	unset (delete) internal variable
\! [COMMAND]	execute command in shell or start interactive shell

Query Buffer

\e [FILE]	edit the query buffer (or file) with external editor
\g [FILE]	send query buffer to server (and results to file or pipe)
\p	show the contents of the query buffer
\r	reset (clear) the query buffer
\s [FILE]	display history or save it to file
\w FILE	write query buffer to file

Input/Output

\echo [STRING]	write string to standard output
\i FILE	execute commands from file
\o [FILE]	send all query results to file or |pipe
\qecho [STRING]	write string to query output stream (see \o)

Informational||

\d [NAME]	describe table, index, sequence, or view
\d{t|i|s|v|S} [PATTERN] (add "+" for more detail)	list tables/indexes/sequences/views/system tables
\da [PATTERN]	list aggregate functions
\db [PATTERN]	list tablespaces (add "+" for more detail)
\dc [PATTERN]	list conversions
\dC	list casts
\dd [PATTERN]	show comment for object
\dD [PATTERN]	list domains
\df [PATTERN]	list functions (add "+" for more detail)
\dg [PATTERN]	list groups
\dn [PATTERN]	list schemas (add "+" for more detail)
\do [NAME]	list operators
\dl	list large objects, same as \lo_list
\dp [PATTERN]	list table, view, and sequence access privileges
\dT [PATTERN]	list data types (add "+" for more detail)
\du [PATTERN]	list users
\l	list all databases (add "+" for more detail)
\z [PATTERN]	list table, view, and sequence access privileges (same as \dp)

Formatting

\a	toggle between unaligned and aligned output mode
\C [STRING]	set table title, or unset if none
\f [STRING]	show or set field separator for unaligned query output
\H	toggle HTML output mode (currently off)
\pset NAME [VALUE]	set table output option (NAME := {format|border|expanded|fieldsep|footer|null| | |numericlocale|recordsep|tuples_only|title|tableattr|pager})
\t	show only rows (currently off)
\T [STRING]	set HTML tag attributes, or unset if none \x toggle expanded output (currently off) Copy, Large Object \copy ... perform SQL COPY with data stream to the client host \lo_export LOBOID FILE \lo_import FILE [COMMENT] \lo_list \lo_unlink LOBOID large object operations

Simple expect ssh example

This expect script would be called from a shell script, and would ssh to the host passed as an argument (argv), perform the command specified, and disconnect. (Thanks, Tiger O.)

#!/usr/bin/expect

set timeout 1
set cmd {uname -a}

spawn ssh root@$argv
expect_after eof { exit 0 }


## interact with SSH
expect "yes/no" { send "yes\r" }
expect "password:" { send "rootpasswd\r" }

expect "# "
send "$cmd\r"
expect "$cmd\r"
expect "(.*)\r"
send "exit\r

120 VAC Outlet / Plug wiring

First of all, don't follow this -- get a qualified electrician to do your work. Doing it yourself, or using this information, may get you or someone else killed. This is my personal reminder note.

The three slots/holes and their attachment points are:

Silver color screws, this is where your grounded leg {aka neutral} wire goes, this by North America standards should be a white wire, however it can be white or gray. (looking at the front face of an outlet, with the ground on bottom, this is the slot/screw on the left).

Equipment grounding screw, this is where the circuit grounding conductor wire will be attached, this on most general purpose circuits will be a bare wire (although an equipment grounding conductor can be bare or green), this bare or green wire is also connected to a grounding screw on the back of the electrical box (if the electrical box in use is metallic)

Brass color screws, this is where the ungrounded leg [aka hot conductor] wire is attached, the most frequently used wire colors are black & red but can be any color in rainbow but white, gray, bare or green.

Do a smoke-test with something unimportant; don't touch it while testing it. Measure potential between the new ground and a known good ground.

Windows Updates from the command line with wuauclt.exe

wuauclt.exe has these parameters (not all are verified. gleaned from various posts on the web.)

/configlist - lists WUAUCLT configuration./installAUclient - Install the correct AU client by force

/installAUclientFromMicrosoft - This option causes the system to contact either the confiugred WSUS server, or Microsoft's WU server, and to reinstall forefully the latest version of the AU client.

/clearlogfile - clears the client update log file

/clearandsavelogfile - saves the current client update log file to a named file, then clears the update log.

/downloadnow - initiates an immiate downoad of any requried update using BITS

/downloanowfast - initiates an initiates an immiate downoad of any requried update using HTTP.

/stopdownload - stops any AU updates being downloaded (either using HTTP, or BITS).

/TestWSUSServer - checks connection with configured WU Server

/DetectNow

/ResetAuthorization

/RunHandlerComServer

/RunStoreAsComServer

/ShowSettingsDialog

/ResetEulas

/DemoUI

/FastCountdown

/FakeData

/FakeReboot

/SkipEulas

Samba with idmap RID on CentOS

For some reason, the CentOS authconfig-tui never works for me out of the box. And there doesn't appear to be correct error handling for it, such as "failed to join domain" -- it acts like everything is okay, and then it doesn't work.

I use the IDMAP_RID option to map user accounts within a single AD to unix user ID's.

Ensure that you have krb5-libs, samba-common, pam_krb5, krb5-workstation, samba-client, and krb5-auth-dialog installed.

Most of these are case-sensitive. The example NETBIOS domain name is "mydomain". Full DNS domain name is "mydomain.com". DC's/KDC's are dc1.mydomain.com, dc2.mydomain.com, etc.

/etc/pam_smb.conf
MYDOMAIN
dc1.mydomain.com
dc2.mydomain.com

Then run "ln -sf system-auth-ac /etc/pam.d/system-auth"
mkdir /var/cache/samba/smb_krb5

/etc/pam.d/system-auth-ac
#%PAM-1.0
# This file is auto-generated.
# User changes will be destroyed the next time authconfig is run.
auth required pam_env.so
auth sufficient pam_unix.so nullok try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth sufficient pam_krb5.so use_first_pass
auth sufficient pam_winbind.so use_first_pass
auth required pam_deny.so

account required pam_unix.so broken_shadow
account sufficient pam_localuser.so
account sufficient pam_succeed_if.so uid < 500 quiet
account [default=bad success=ok user_unknown=ignore] pam_krb5.so
account [default=bad success=ok user_unknown=ignore] pam_winbind.so
account required pam_permit.so
assword requisite pam_cracklib.so try_first_pass retry=3
password sufficient pam_unix.so md5 shadow nullok try_first_pass use_authtok
password sufficient pam_krb5.so use_authtok
password sufficient pam_winbind.so use_authtok
password required pam_deny.so

session optional pam_keyinit.so revoke
session required pam_limits.so
session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session required pam_unix.so
session optional pam_krb5.so


/etc/nsswitch.conf
passwd: files winbind
shadow: files winbind
group: files winbind
hosts: files dns
bootparams: nisplus [NOTFOUND=return] files
ethers: files
netmasks: files
networks: files
protocols: files
rpc: files
services: files
netgroup: files
publickey: nisplus
automount: files
aliases: files nisplus
publickey: nisplus
automount: files nisplus
aliases: files nisplus

/etc/krb.conf
MYDOMAIN.COM
MYDOMAIN.COM dc1.mydomain.com:88
MyDOMAIN.COM dc2.mydomain.com:88
MYDOMAIN.COM dc1.mydomain.com:749 admin server

/etc/krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = MYDOMAIN.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
forwardable = yes

[realms]
MYDOMAIN.COM = {
kdc = dc1.mydomain.com:88
kdc = dc2.mydomain.com:88
admin_server = dc1.mydomain.com:749
}

[domain_realm]
domain.com = MYDOMAIN.COM
.domain.com = MYDOMAIN.COM

[kdc]
profile = /var/kerberos/krb5kdc/kdc.conf

[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}

/etc/samba/smb.conf
#=======Global Settings =================
[global]

# workgroup = NT-Domain-Name or Workgroup-Name, eg: MIDEARTH
workgroup = MYDOMAIN
netbios name = FILESERVER
realm = MYDOMAIN.COM
# server string is the equivalent of the NT Description field
server string = File Server
# Security mode. Defines in which mode Samba will operate
security = ADS
allow trusted domains = No
idmap backend = rid:DOMAIN=1000000-10000000
idmap uid = 1000000-10000000
idmap gid = 1000000-10000000
template shell = /sbin/nologin
winbind use default domain = Yes
winbind enum users = Yes
winbind enum groups = Yes
winbind nested groups = Yes
smb ports = 139
# Restrict what subnets can access this server
hosts allow = 192.168.1. 192.168.2. 192.168.3. 192.168.4. 192.168.5. 192.168.0. 127.

# If you want to automatically load your printer list rather
# than setting them up individually then you'll need this
load printers = no
domain master = no
preferred master = no
domain logons = no
wins support = no
wins proxy = no
dns proxy = no
password server = dc1.mydomain.com
# SHARE DEFINITIONS
[users]
path = /var/export/users
comment = user home dirs
public = no
writeable = yes
browseable = no
guest ok = no
printable = no
[team]
path = /var/export/team
comment = Team and Departmental Share
public = no
writeable = yes
browseable = yes
guest ok = no
write list = @Domain\ Users
valid users = +"DOMAIN\Domain Users" jdoe
#invalid users = @acc_T_Drive_Deny mdc
printable = no


Now, run "ntpdate dc1.mydomain.com", then configure and start ntpd (configure it to sync time from your DC's). Use ntpd -q and look for an asterisk to know when it has synchronized to the DC; it may take a few minutes. If your system clock gets more than 5 minutes out of sync with the DC's, then kerberos (and your authentication) will stop working.

TESTING
To test Kerberos functionality for authentication, run kinit " where is a user name on your AD domain.

To test ntpd functionality, run ntpd -q and look for an asterisk next to your DC.

To test that winbind is getting user and group info from AD, and that the system can use it, run "getent passwd" and "getent groups", and you should see your AD users and groups in there.

Note: for VMware server to work, you'll need a manual password entry for any user who will run vmware to be in /etc/passwd.