Vi/Vim Tips

These are some things I always seem to forget and have to look up again; they don't seem to all be in the same place anywhere, or stated as succinctly.  Some are VIM only.

(type :help


m  : set a mark at the current position in this file(a-z) (per-file)
m  : set a *global* mark at the current position, reachable from different files.
:marks  : list marks
:marks aF : list marks a, F


(precede any of these with a number to repeat it that many times)
` : move to the position of a mark in this file (see "Marking")
' : move to the line of a mark in this file (see "Marking")
` : move to the position of a (global) mark in another file (see "Marking")
' : move to the line of a (global) mark in another file (see "Marking")
h   : move one character left
l   : move one character right
k   : move one character/row up
j   : move one character/row down
w   : move to beginning of next word after punctuation
W   : move to beginning of next word after whitespace
b   : move to beginning of previous word after punctuation
B   : move to beginning of previous word after whitespace
e   : move to end of this word
E   : move to the end of this word before whitespace
0   : move to beginning of the current line (column 0)
^   : move to the non-whitespace beginning of the current line
_   : move to the non-whitespace beginning of the current line with count (e.g. 5_)
$   : move to the end of a line
g_  : move to the last non-whitespace character of the line, with count (e.g. 5g_)
gg  : move to first line
G   : move to last line
ng  : move to line number "n"
H   : move to the first line of the screen (home)
M   : move to the middle line of the screen
L   : move to the last line of the screen
z   : re-center the screen on the current line (with the cursor)
zt  : re-center the screen with the current line (with the cursor) at the top
zb  : re-center the screen with the current line (with the cursor) at the top
ctrl-D : move a half-page down
ctrl-U : move a half-page up
ctrl-B : page up (back)
ctrl-F : page down (forward)
ctrl-o : return to last cursor position
ctrl-i : go to next cursor position
%    : jump to the matching (){}[]

Basic Editing

u    : undo last edit
U    : return last changed line to its former state
ctrl-R : redo last undone edit
cw   : change text of current word
c : replace one character
c`: change text from current position to position of marker
c': change text from current line to line of marker


p    : put/paste following this line
P    : put/paste preceding this line
dd   : delete/cut the current line
D    : delete/cut from current character to the end of the line
yy   : copy (yank) the current line
y$   : copy from current position to the end of the line
*y$  : copy from current position to the end of the line, place in system clipboard
*p   : paste from system clipboard to following line

d`: delete from current position to position of marker a-z
d': delete from current line to line of marker a-z
y`: copy (yank) text from current position to position of marker
y': copy (yank) text from current line to line of marker

Search / Replace

/string : search forward for "string"
?string : search backward for "string"
n    : find next occurrence of the search string (in the same direction as the search)
N    : find previous occurrence of the search string
*    : find the next occurrence of the word currently under the cursor
#    : find the previous occurrence of the word currently under the cursor
g*   : find the next occurrence of the search pattern under the cursor
g#   : find the previous occurrence of the search pattern under the cursor

Change Case

~    : Changes the case of current character 
guu  : Change current line from upper to lower. 
gUU  : Change current LINE from lower to upper. 
guw  : Change to end of current WORD from upper to lower. 
guaw : Change all of current WORD to lower. 
gUw  : Change to end of current WORD from lower to upper. 
gUaw : Change all of current WORD to upper. 
g~~  : Invert case to entire line

Global settings

(many of these can be permanently placed in your ~/.vimrc file)
set tabstop=4  : set how wide a "tab" is




Create Postgres read-only user (pre 9.x)

Here’s how I created the user; due to limitations in postgres 8.1, this user will not have access to any new tables (such as may be created during an upgrade of the application); the process is much easier than this for postgres 9.1+.

(Credit to this helpful user http://stackoverflow.com/a/762649 ; I just want to show it in action and spread the love. )

First, connect to postgres, then list databases, then connect to the correct one, then create the DB user.
[root@mycomputer ~]# su - postgres
-bash-3.2$ rpm -qa | grep postgres
-bash-3.2$ psql
Welcome to psql 8.1.23, the PostgreSQL interactive terminal.

Type:  \copyright for distribution terms
       \h for help with SQL commands
       \? for help with psql commands
       \g or terminate with semicolon to execute query
       \q to quit

postgres=# \list
         List of databases
    Name    |   Owner    | Encoding
testdb | testdb | UTF8
postgres   | postgres   | UTF8
template0  | postgres   | UTF8
template1  | postgres   | UTF8
(4 rows)

postgres=# \c testdb
You are now connected to database "testdb".
testdb=# CREATE USER testdbread WITH password 'supersecretpassword';
testdb=# GRANT USAGE ON SCHEMA public TO testdbread;

# This command creates a list of GRANT commands that you can copy and paste to grant access to existing tables:
testdb=# select 'GRANT SELECT ON ' || relname || ' TO testdbread;' FROM pg_class JOIN pg_namespace ON pg_namespace.oid = pg_class.relnamespace WHERE nspname = 'public' AND relkind IN ('r', 'v', 'S');
GRANT SELECT ON usermigration TO testdbread;
GRANT SELECT ON plugindata TO testdbread;
GRANT SELECT ON os_propertyentry TO testdbread;
GRANT SELECT ON os_user TO testdbread;
GRANT SELECT ON attachments TO
(5 rows)
# Now, copy and paste those lines, then exit:

testdb=# GRANT SELECT ON usermigration TO testdbread;
testdb=# GRANT SELECT ON plugindata TO testdbread;

testdb=# GRANT SELECT ON os_propertyentry TO testdbread;

testdb=# GRANT SELECT ON os_user TO testdbread;
testdb=# GRANT SELECT ON attachments TO testdbread;
testdb=# exit
testdb-# \q
-bash-3.2$ logout
[root@mycomputer ~]#


Protect and access your passwords anywhere

Web sites are regularly compromised and your data may be compromised.  Server applications have as-yet-undiscovered weaknesses, as do even security appliances and encryption protocols.  The Heartbleed vulnerability with the very widely used OpenSSL was recently made public.  The details on how and even whether it may have been used to compromise data are not as clear to me.  However, we know that the vulnerability can allow sensitive data to be stolen from devices that use it for encryption.

In any case, if you haven't rolled your passwords in a while, this could be a good time to do so.  If you're doing so, it should also be a good time to finally start protecting your online and financial identities in this well-connected world the same way you lock the door on your home and perhaps have an alarm on your car.

Others may/will disagree, but here's what I recommend, as a way to (1) always have strong passwords, (2) reduce risk of a compromise of one account meaning other things get compromised, and (3) ensure that you can always get to your passwords in a pinch, any time, from nearly any device.

  1. Use a local storage password keeper program, one that never uses unencrypted temporary files and has sensible semantics for automatically backing itself up and locking itself.  I like Password Safe, for example. Download the latest here: http://sourceforge.net/projects/passwordsafe/files/passwordsafe/  (supports Windows, Linux, Android, MacOS X, iOS) 
  2. Use a free backup program and service from a reputable (publicly traded, with a lot to lose if they abuse your data,) to sync your encrypted password safe and ensure it survives local hard drive failures.  I use DropBox.  Dropbox lets you have a "dropbox" folder on your local computer (Windows, Linux, Android, MacOS X, iOS) that is automatically backed up to a cloud, and makes the files accessible from other computers (e.g., between yours, mom's laptop, and home PC; even via the web, and from Android and iOS phones).  Once you have run password safe and "saved" your password safe to your "Dropbox" folder, it is backed up automatically.
  3. With two exceptions, all of your passwords will be completely random; only your password keeper program and your password safe will have phrases that you can remember, that nobody else could guess, and that are different from each other.
    1. I suggest picking a sentence that only you would know for your password, something from your childhood like: "Oh, how I loved camp chatta-wookie as a child." (If you thought to use that exact phrase, repent and sit in the corner for 5.)
    2. Better yet, let your spouse choose a phrase that you can both remember. Like a sentence about a great memory -- you get the idea.  Then you can share the same password safe.  If something happens to one of you, the other can still log in to your sites to take care of business.
    3. You could use a favorite line from a book, but you need to change it in a way that someone else would not use or guess the exact same phrase; e.g., don't use "For God so loved the world" or "Et tu, Brute?", because those are very common phrases.
  4. Create an account with your online backup/sync provider using the pass phrase advice in the last step.  You will need to be able to recall this if you ever want to access it from a computer other than your regular PC/laptop, or if your local hard drive fails.  This is the one password that you may write down somewhere and keep in your safe. Compromise of this phrase will allow someone to get at your synced files, but they won't be able to get your passwords -- those will be encrypted using a different password.
  5. Start up the password safe program and assign a different passphrase than the one you used for your online backup/sync provider.  If you forget this pass phrase, you're out of luck.  Save your new password safe data file to the location that your backup provider uses. For drop box, that would be a "Dropbox" folder under your home directory or documents folder.
  6. Now, go to each site where you have an account and change the password, recording it in the password safe program and following this method:
    1. For Title, use a name of the service so that you can find it quickly in the list. When your list grows large, you'll want something that is meaningful
    2. For Username, use the exact username as you use it to log in.
    3. For password, click "Generate".  For details on how to configure Password Safe to be naturally stronger, see below.  If you're not going to use strong, unique (if not random) passwords, then nothing else you do will matter too terribly much, and you have left your windows unlocked and inviting.
    4. Copy and paste the generated password into your web site in the change password / new password field.
    5. URL: specify the web site URL, if it's a web site. This will help you to remember how to get to the exact site, or to search your list of passwords more easily.
    6. Use "Notes" to record stuff you'd need to remember. You might record the date you opened the account, or answers to security questions (see below on security question).
    7. Click "OK" to apply.  I like to go into password safe and configure it to automatically save every time I change a password. I also like to configure it to *not* use the system tray, so it actually exits and closes when I click the close button.
  7. Thoughts on passwords
    1. Always let let your password safe program randomly generate passwords for every site where you register.   Perhaps the only exceptions would be your main email account, your password safe "combination", and your data sync/backup site.  Set those to other sentences that you will always remember. 
    2. Never, ever use the same password twice.
    3. Using random and unique passwords ensures that if a person gets a password or access to one site, they'll have a harder time getting access to another site. Consider that some IT staff and related functions may be able to actually see your passwords that you used to register for sites at companies you deal with; if you use the same passwords, an unscrupulous IT staffer could then simply guess what sites you might use in order to gain access to other accounts and steal your data or impersonate you.
    4. Ensure that the passwords are long and complex (10+ characters, including alpha, num, and punctuation where the site permits it).  In Password Safe, you configure the "password policy" using the Manage menu --> Options --> Password Policy.
    5. Never write down your passwords elsewhere for any site.  Even silly sites.  Compromising silly sites is one way for someone to socially engineer their way into assuming parts of your identity or learning more about you to compromise the rest.  An exception would be to put the password in your strong home safe (not in a locking drawer that can easily be broken into).
    6. Never access services like these from a public or shared computer. Never.
  8. Thoughts on security questions:
    1. Generally speaking, security questions are a bad idea: they provide in some circumstances a backdoor to bypass the best passwords; and it is too easy for other people to socially engineer in order to take advantage of security questions.
    2. How many people know the name of your pet, your first school, the street on which you grew up, your teachers, mother's maiden name?  Too many. Much of it is public knowledge; much can be gleaned by casual strangers from social networking sites.
    3. My recommendation for security questions?  Give bogus, random-ish answers, and store those questions and answers in the "Notes" section for that account in your password safe.  If the answer is false and unrelated to real life, it will be much harder for a ne'er do well (old boyfriend, identity thief, etc.) to provide those answers to gain access to an account.
  9. Thoughts on the complexity, and the rule of KISS:
    1. While I understand that this is not as easy as using a password sync service, I still prefer this methodology, in part because I do not necessarily trust any vendor with a single point of access.
    2. Because password safe encrypts the file locally, and that encrypted file is synced to a private dropbox account, there is not a single person that could potentially have access to my data: even if password safe's encryption is compromised (and it will be, like everything), then someone would still have to gain access to the encrypted datafile on my personal dropbox account or on my personal computer. 
Now, if you've done all of this, then you are more protected than otherwise against password guessing and other attacks, and you can get to your passwords from any computer, even an android phone, as long as you remember your password safe account (and your dropbox password, if you don't have any other computer).  

You'll never keep the NSA away from your passwords, data, and accounts, but this should keep organized crime and casual miscreants away.

Using this method, I even have secure access to my passwords from my smart phone when I need them.  And I don't have to actually remember but a couple of passwords.

I welcome your thoughts and critiques. Everybody has blinders, perhaps you've found a problem with this method?


Grow root partition and filesystem in Ubuntu and Linux Mint

Here are the steps to grow the root filesystem on an existing system that does NOT use LVM:
  1. First, use LVM; it makes resizing filesystems nearly trivial, robust, without any real downsides.  Just another way that Ubuntu is not enterprise-worthy.  But, we don’t have LVM on these systems’s as they are….
  2. Second, before doing any operations on your partitions, always perform and test a backup (to a separate system!) of your data. You may make your system unbootable and/or nuke all of your data!
  3. The system I performed these steps on was partitioned thus:
    1. /dev/sda – 7GB – /
    2. /dev/sda2 – remainder – Extended partition
    3. /dev/sda5 – swap – partition within the extended partition.
  4. So, to grow “/”, we will become root, delete the swap partition and the extended partition, grow /, and then re-create the swap partition.  Adjust the steps below according to the partition numbers and layout of your particular system:
    1. Disable swap:
      1. swapoff /dev/sda5
    2. Delete and re-create partitions as appropriate.
      1. fdisk /dev/sda
      2. print out partition information (p)
      3. Delete partition 5 (d – 5)
      4. Delete partition 2 (d – 2)
      5. Delete partition 1 (d)
      6. Create partition 1 (n – p – 1)
        1. It must start on the exact same sector as before (as seen in the print command)
        2. It must end on a sector higher than it did before.  Num_GB*1024*1024*2 = ending sector
      7. Create partition 2 (n – p – 2)
      8. Change partition 2 to type “Linux Swap” (t – 2 – 82 )
      9. Activate partition 1 to make it bootable (a - 1)
      10. Double-check everything.
      11. Exit (w)
    3. recreate the swap partition, using a label:
      1. mkswap -L swap /dev/sda2
    4. add a label to the root filesystem
      1. e2label /dev/sda1  /   (in centos, it’s e4label)
    5. Fix up /etc/fstab
      1. Fix the “swap” line to use LABEL=swap instead of UUID=
      2. Fix the “/” line to use LABEL=/ instead of UUID=
    6. swapon /dev/sda2
    7. Fix up /boot/grub/grub.cfg
      1. Either: Find the “linux” line for the menu option you will boot, change UUID= to LABEL=/
      2. Or: update-grub
    8. Re-create the initrd
      1. update-initramfs -u -k 3.2.0-38-generic
    9. Reboot the VM. *Cross your fingers!* This is your moment of truth.
    10. Grow the root filesystem
      1. resize2fs /dev/sda1  (resize4fs on CentOS, I believe).
    11. If it didn't boot, then that's what you get for biting off more than you could chew, and for choosing a distro that doesn't leverage LVM.  Boot off your Ubuntu/Mint install disk, and copy off your data to a USB disk, and start over. (...but you DID back up your data anyway, right?)


Automatically delete old NetApp snapshots left by backups

Backup software may sometimes leave behind a snapshot.  This can increase the space consumption a lot over time.

Here's a script that can be run against NetApp filers to clean up those "stale" backups.  Note that in my case, I configured CommVault to name the snapshots with the string "snapshot_for_backup", though the default is just "ndmp".  You may change that as needed.


# This script is to look for snaps that are left over from backups, that # are no longer in use, and delete them.
# This script requires two parameters:
# snap_cleanup     (list or delete all stale snaps on the specified servers)


# This function receives a volume name as a parameter, and returns # the number of snaps that are elligible for deletion, defined by being:
#                             1. Having a certain string in the snapshot name;
#                             2. Not marked as "busy"
function CountStaleSnaps ()
                local VolToCheck=$1
                local StaleSnaps=`$SshCmd "snap list $VolToCheck" | grep $SnapshotString | grep -v "busy" | wc -l`
                if [ $? -ne 0 ]; then
                                echo "getting count of elligible snapshots returned an error"
                                exit 1
                } else
                                echo $StaleSnaps
                } fi

# Function GetStaleSnapNames
# This function creates, given a volume name, an array of snapshots that are # candidates for deletion.
# Parameters
#             1. volume to check
function GetStaleSnapNames ()
                local Volume=$1
                StaleSnapNames=( $($SshCmd "snap list $Volume" | grep $SnapshotString| grep -v busy | cut -c 39- | cut -f 1 -d " ") )
                if [ $? -ne 0 ]; then
                                echo "getting names of elligible snapshots returned an error"
                                exit 1
                } else
                                printf -- '%s\n' "${StaleSnapNames[@]}"
                } fi

# Function DelStaleSnaps
# This function deletes stale snaps for the Volume name passed to it.
# Parameters
#  1. volume
#  2. snapshot_name
function DelStaleSnaps ()
                local Volume=$1
                local SnapToDelete=$2
                $SshCmd "snap delete $Volume $SnapToDelete"
                if [ $? -ne 0 ]; then
                                echo "error deleting snapshot $Volume:$SnapToDelete"
                                exit 1
                } else
                                echo "successfully deleted snapshot $Volume:$SnapToDelete"
                } fi

# Parse command line parameters
case $2 in
                                echo "ERROR: you must specify as a second parameter a host name on which you want to delete snapshots."
case $1 in
                                echo "ERROR: invalid operation specified on command line.  Please specify either 'list' or 'delete' followed by the servername on which you want to delete the snapshots."
                                exit 1

SshCmd="/usr/bin/ssh -i $SshIdentityFile $NasUser@$NasName"

VolumesToCheck=( `$SshCmd "vol status -b" | cut -f 1 -d " " | egrep -v "Volume|-----"` )

for CurrentVol in ${VolumesToCheck[@]}; do {
                echo -n "checking $NasName:$CurrentVol... "
                StaleSnaps=`CountStaleSnaps $CurrentVol`
                echo $StaleSnaps            
                if [ $Operation = list ]; then
                                if [ $StaleSnaps -ne 0 ]; then
                                                GetStaleSnapNames $CurrentVol |  awk '{ print "    " $1 }'
                elif [ $Operation = delete ]; then
                                GetStaleSnapNames $CurrentVol
                                ArrayOfSnaps=( $(GetStaleSnapNames $CurrentVol) )
                                for TargetSnap in `printf -- '%s\n' "${ArrayOfSnaps[@]}"`; do
                                                                DelStaleSnaps $CurrentVol $TargetSnap
}; done


Linux bash shell options parsing

Normally, command line parameters come in as separate, positional values, and may be referenced as:
  • $# - the number of command line arguments (positional parameters)
  • $* - all positional parameters expressed as a single string
  • $@ - all positional parameters, but with each as a quoted string (each positional parameter is intact and presented as a quoted string)
  • $0 - (the base name of the script itself)
  • $1 - The first positional parameter
  • $2 - The second positional parameter, and $3, $4, etc.  Starting with 10, they must be expressed as ${10}, ${11}, etc.
Sometimes a variable is passed from a wrapper script to a child script where the positional parameter is in fact several parameters that should be parsed separately.

Here, we test for that case, and if found, we peel off the first parameter within that group of strings, and assign the remaining parameters in the group of springs:

# from the front, and make the rest of the data be the options passed to rsync.
if [ $# -eq 1 ]; then
   unset Params[0]
} fi


Manipulate windows firewall from CLI

The windows firewall UI is a bit cumbersome.  This method will let you set up a host reliably to be secure, and to allow only inbound traffic that you want.

Enable firewall in all profiles (home / domain / public)
netsh advfirewall set allprofiles state on

By default, deny all inbound and allow all outbound traffic
netsh advfirewall set allprofiles firewallpolicy blockinbound,allowoutbound

Add rule to allow SMTP traffic inbound to a specific port from a specific network range
netsh advfirewall firewall add rule name="Allow Inbound TCP/25 from SMTP relay hosts" protocol=TCP dir=in localport=25 action=allow remoteip=

Add rule to allow all HTTP and HTTPS traffic inbound
netsh advfirewall firewall add rule name="Allow Inbound TCP/80 from everywhere" protocol=TCP dir=in localport=80 action=allow


Upgraded ESXi host has errors, won't permit some operations

I have some ESXi hosts that were 4.x, then upgraded to 5, then 5.1 .  As of today, they have errors that complain about the HA agent being broken.  They also won't let me enable/disable the SSH service nor modify the firewall configuration.

The error for the SSH service starts with 'Call "HostFirewallSystem.DisableRuleset" for object' .

These two articles provide the answers. After following the first step, I no longer get the error about SSH and firewall.

After following the second article, my HA agent is no longer complaining.



Boiling the steps down, I have
  1. Fix SSH and firewall
    1. enable ssh on the host (if not already enabled); this is under host --> configuration --> security profile
    2. ssh to the host
    3. cd /etc/vmware/service
    4. cp services.xml service.xml.bak
    5. vi services.xml (remove the line with "sshServer")
    6. esxcli network firewall refresh
    7. disable ssh on the host (but don't close your connection if you're going to continue with the next process)
  2. Restart the ESXi host agents
    1. On the host, run this command: /sbin/services.sh restart
    2. this step takes a few minutes to complete.


SNMP OIDs for temperature monitoring

...This is for anyone looking for these things in one place, with the scale and units. ( extracted from http://wleibzon.bol.ucla.edu/nagios/plugins/check_snmp_temperature.pl )
  • Dell (10C)
  • Cisco (C)
    • fans:
  •  juniper (C)
  • HP (C)
  • alteon (C)
    • Rear Left Sensor -
    • Rear Middle Sensor -
    • Front Middle Sensor -
    • Front Right Sensor -
  • baytech PDU (10C)
  • Linux lmsensors (1000C)
  • APC temperature
  • APC humidity 
  • HP switch
    • temperature:
    • fan:


Tweaks for Linux Mint

These were started with Linux Mint 14.1 Cinnamon

Menu bar shows up on the wrong screen:

Do this to find out the name of the displays:
sudo xrand -q
Do this to make the menu show up on one display or the other, for example:
sudo xrand --output VGA1 --primary

Sound doesn't come out through the correct device (speaker, headphone, etc.):

run these:
sudo apt-get install alsa-base alsa-utils pavucontrol pavumeter paman
...then in alsamixer, select the desired output "card", then ensure that "automute" is set to disabled or some other device.

I want to watch Netflix on Mint:

Run these commands:
sudo apt-add-repository ppa:ehoover/compholio
sudo apt-get install netflix-desktop

Display locks up with my NVidia card:

During install, select "compatibility mode".

After install, before logging in, select session type "Cinnamon 2D", then
sudo apt-get install nvidia-current nvidia-settings
sudo reboot