About Tom Ryder

Systems administrator and web developer living in Palmerston North, New Zealand.

Special characters in Vim

Particularly when editing documents for human consumption rather than code, it’s often necessary to enter special characters into a document that can’t otherwise be produced by a single key press:

  • Letters with diacritical marks like ä, é, and ô — Vim refers to these as digraphs — a particular problem when using a US keyboard layout
  • Unicode characters like typographic dashes or copyright symbols ©, or other symbols from the multi-byte portion of the UTF-8 character set (including foreign languages)
  • Literal control characters like <Tab>

Vim has a method for inserting each of these within the editor, rather than having to copy-paste them from another document. We won’t discuss Vim’s alternative multibyte input methods here, and will assume that you’re using a keyboard with a US or UK layout, and predominantly type in English – apologies to international readers, but I do not have another type of keyboard to test this out!

Some of the following assumes that you’re using Vim in a UTF-8 capable terminal, and with the encoding option in your .vimrc set to utf-8, which is highly recommended for the vast majority of editing requirements:

set encoding=utf-8

It also assumes that your font is capable of displaying all of the characters concerned; monospace fonts with workable symbol coverage include Consolas, Inconsolata, and Ubuntu Mono.

Digraphs

Vim has a special shorthand for entering characters with diacritical marks. If you need some familiar variant of a Latin alphabet character with a diacritical mark or embellishment, it’s likely you’ll be able to input it with the digraph system. It also has support for some other sometimes-needed characters like thorn Þ and eszett ß, and Cyrillic characters.

Digraph input is started in insert or command mode (but not normal mode) by pressing Ctrl-k, then two printable characters in succession; the first is often the “base” form of the letter, and the second denotes the appropriate embellishment.

Some simple examples that might occasionally be needed for English speakers to correctly type one of the language’s many “loan words”:

  • Ctrl-k c , -> ç
  • Ctrl-k e ' -> é
  • Ctrl-k o ^ -> ô
  • Ctrl-k a ! -> à
  • Ctrl-k u : -> ü
  • Ctrl-k = e ->

This is just a small sample; Vim has support for a great many digraphs. Take a look at the relevant section of the documentation for a complete treatment of the feature. You can also type :digraphs within Vim to get a complete list of digraphs — several screenfuls of them!

Note that you can enter all of these characters using the Unicode mode discussed later in this article as well; two-character mnemonic digraphs simply happen to be easier to remember than four-digit codes.

Unicode characters

For characters not covered in the digraph set, you can also enter unicode characters by referring to their code page number. In insert or command mode (but not normal mode) this is done by typing Ctrl-v and then u, followed by the hexadecimal number. Some potentially useful examples:

  • Ctrl-v u 2018 -> , a LEFT SINGLE QUOTATION MARK
  • Ctrl-v u 2019 -> , a RIGHT SINGLE QUOTATION MARK
  • Ctrl-v u 2014 -> , an EM DASH
  • Ctrl-v u 00a9 -> ©, a COPYRIGHT SIGN

These are handy in some cases when writing HTML documents, as an alternative to using HTML entities like &mdash; or &copy;. An exhaustive summary of these characters and their codes is available on the Unicode website.

Other non-printable characters

The unicode character input method is actually a specialised case of inputting literal characters with a Ctrl-v prefix. We can input other non-printable and control characters using this prefix:

  • Ctrl-v <Enter> -> ^M
  • Ctrl-v <Tab> -> ^I

This is sometimes handy when conforming to someone else’s tab style, and can also be handy when searching for characters literally in searches.

SSH, SOCKS, and cURL

Port forwarding using SSH tunnels is a convenient way to circumvent well-intentioned firewall rules, or to access resources on otherwise unaddressable networks, particularly those behind NAT (with addresses such as 192.168.0.1).

However, it has a shortcoming in that it only allows us to address a specific host and port on the remote end of the connection; if we forward a local port to machine A on the remote subnet, we can’t also reach machine B unless we forward another port. Fetching documents from a single server therefore works just fine, but browsing multiple resources over the endpoint is a hassle.

The proper way to do this, if possible, is to have a VPN connection into the appropriate network, whether via a virtual interface or a network route through an IPsec tunnel. In cases where this isn’t possible or practicable, we can use a SOCKS proxy set up via an SSH connection to delegate all kinds of network connections through a remote machine, using its exact network stack, provided our client application supports it.

Being command-line junkies, we’ll show how to set the tunnel up with ssh and to retrieve resources on it via curl, but of course graphical browsers are able to use SOCKS proxies as well.

As an added benefit, using this for browsing implicitly encrypts all of the traffic up to the remote endpoint of the SSH connection, including the addresses of the machines you’re contacting; it’s thus a useful way to protect unencrypted traffic from snoopers on your local network, or to circumvent firewall policies.

Establishing the tunnel

First of all we’ll make an SSH connection to the machine we’d like to act as a SOCKS proxy, which has access to the network services that we don’t. Perhaps it’s the only publically addressable machine in the network.

$ ssh -fN -D localhost:8001 remote.example.com

In this example, we’re backgrounding the connection immediately with -f, and explicitly saying we don’t intend to run a command or shell with -N. We’re only interested in establishing the tunnel.

Of course, if you do want a shell as well, you can leave these options out:

$ ssh -D localhost:8001 remote.example.com

If the tunnel setup fails, check that AllowTcpForwarding is set to yes in /etc/ssh/sshd_config on the remote machine:

AllowTcpForwarding yes

Note that in both cases we use localhost rather than 127.0.0.1, in order to establish both IPv4 and IPv6 sockets if appropriate.

We can then check that the tunnel is established with ss on Linux:

# ss dst :8001
State      Recv-Q Send-Q   Local Address:Port       Peer Address:Port
ESTAB      0      0            127.0.0.1:45666         127.0.0.1:8001
ESTAB      0      0            127.0.0.1:45656         127.0.0.1:8001
ESTAB      0      0            127.0.0.1:45654         127.0.0.1:8001

Requesting documents

Now that we have a SOCKS proxy running on the far end of the tunnel, we can use it to retrieve documents from some of the servers that are otherwise inaccessible. For example, when we were trying to run this from the client side, we found it wouldn’t work:

$ curl http://private.example/contacts.html
curl: (6) Couldn't resolve host 'private.example'

This is because the example subnet is on a remote and unroutable LAN. If its name comes from a private DNS server, we may not even be able to resolve its address, let alone retrieve the document.

We can fix both problems with our local SOCKS proxy, by pointing curl to it with its --proxy option:

$ curl --proxy socks5h://localhost:8001 http://private.example/contacts.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
    <head>
        <title>Contacts</title>
...

Older versions of curl may need to use the --socks5-hostname option:

$ curl --socks5-hostname localhost:8001 http://private.example/contacts.html

This not only tunnels our HTTP request through to remote.example.com and returns any response, it does the DNS lookup on the other end too. This means we can not only retrieve documents from remote servers, we can resolve their hostnames too, even if our client side can’t contact the appropriate DNS server on its own. This is what the h suffix does in the socks5h:// URI syntax above.

We can configure graphical web browsers to use the SOCKS proxy in the same way, optionally including DNS resolution:

Browsers are not the only application that can use SOCKS proxies; many IM clients such as Pidgin and Bitlbee can use them too, for example.

Making things more permanent

If this all works for you and you’d like to set up the SOCKS proxy on the far end each time you connect, you can add it to your ssh_config file in $HOME/.ssh/config:

Host remote.example.com
    DynamicForward localhost:8001

With this done, you should only need to type the hostname of the machine to get a shell and to set up the dynamic forward in the background:

$ ssh remote.example.com

Advanced Vim registers

Registers in Vim are best thought of as scratch spaces for text, some of which are automatically filled by the editor in response to certain actions. Learning how to use registers fluently has a lot of subtle benefits, although it takes some getting used to because the syntax for using them is a little awkward.

If you’re reasonably fluent with Vim by now, it’s likely you’re already familiar with the basic usage of the 26 named registers, corresponding to the letters of the alphabet. These are commonly used for recording macros; for example, to record a series of keystrokes into register a, you might start recording with qa, and finish with q; your keystrokes could then be executed with @a.

Similarly, we can store text from the buffer itself rather than commands in these registers, by prepending "a to any command which uses a register, such as the c, d, and y commands:

  • "ayy — Read current line into register a.
  • "bP — Paste contents of register b above current line.
  • "cc3w — Change three words, putting the previous three words into register c.

Like many things in Vim, there’s a great deal more functionality to registers for those willing to explore.

Note that here I’ll be specifically ignoring the *, +, and ~ registers; that’s another post about the generally unpleasant business of making Vim play nice with system clipboards. Instead, I’ll be focussing on stuff that only applies within a Vim session. All of this is documented in :help registers.

Capital registers

Yanking and deleting text into registers normally replaces the previous contents of that register. In some cases it would be preferable to append to a register, for example while cherry-picking different lines from the file to be pasted elsewhere. This can be done by simply capitalizing the name of the register as it’s referenced:

  • "ayyReplace the contents of register a with the current line.
  • "AyyAppend the current line to register a.

This works for any context in which an alphabetical register can be used. Similarly, to append to a macro already recorded in register a, we can start recording with qA to add more keystrokes to it.

Viewing register contents

A good way to start getting a feel for how all the other registers work is to view a list of them with their contents during an editing session with :registers. This will show the contents of any register used in the editing session. It might look something like this, a little inscrutable at first:

:registers
--- Registers ---
""   Note that much of it includes
"0   execut
"1   ^J^J
"2   16 Oct (2 days ago)^J^Jto Jeff, Alan ^JHi Jeff (cc Alan);^J^JPlease 
"3   <?php^Jheader("Content-Type: text/plain; charset=utf-8");^J?>^J.^J
"4   ^J
"5   Business-+InternationalTrade-TelegraphicTransfers-ReceivingInternati
"6   ../^J
"7       diff = auto^J    status = auto^J    branch = auto^J    interacti
"8   ^J[color]^J    ui = auto^J    diff = auto^J    status = auto^J    br
"9       ui = true^J
"a    escrow
"b   03wdei^R=2012-^R"^M^[0j
"c   a
"e   dui{<80>kb^[^[
"g   ^[gqqJgqqjkV>JgqqJV>^[Gkkkjohttp://tldp.org/LDP/abs/html/^[I[4]: ^[k
"h   ^[^Wh:w^Mgg:w^M^L:w^Mjk/src^Mllhh
"j   jjjkkkA Goo<80>kb<80>kb<80>kbThis one is good pio<80>kbped through a
"-   Note that much of it includes
".    OIt<80>kb<80>kb<80>kbIt might looks <80>kb<80>kb something like thi
":   register
"%   advanced-vim-registers.markdown
"/   Frij

The first column contains the name of the register, and the second its contents. The contents of any of these registers can be pasted into the buffer with "ap, where a is the name of any of them. Note that there are considerably more registers than just the named alphabetical ones mentioned above.

Unnamed register

The unnamed register is special in that it’s always written to in operations, no matter whether you specified another register or not. Thus if you delete a line with dd, the line’s contents are put into the unnamed register; if you delete it with "add, the line’s contents are put into both the unnamed register and into register a.

If you need to explicitly reference the contents of this register, you can use ", meaning you’d reference it by tapping " twice: "". One handy application for this is that you can yank text into the unnamed register and execute it directly as a macro with @".

Man, and you thought Perl looked like line noise.

Black hole register

Another simple register worth mentioning is the black hole register, referenced with "_. This register is special in that everything written to it is discarded. It’s the /dev/null of the Vim world; you can put your all into it, and it’ll never give anything back. A pretty toxic relationship.

This may not seem immediately useful, but it does come in handy when running an operation that you don’t want to clobber the existing contents of the unnamed register. For example, if you deleted three lines into the unnamed register with 3dd with the intent of pasting them elsewhere with p, but you wanted to delete another line before doing so, you could do that with "_dd; line gone, and no harm done.

Numbered registers

The read-only registers 0 through 9 are your “historical record” registers. The register 0 will always contain the most recently yanked text, but never deleted text; this is handy for performing a yank operation, at least one delete operation, and then pasting the text originally yanked with "0p.

The registers 1 through 9 are for deleted text, with "1 referencing the most recently deleted text, "2 the text deleted before that, and so on up to "9.

The small delete register

This read-only register, referenced by "-, stores any text that you deleted or changed that was less than one line in length, unless you specifically did so into some other named register. So if you just deleted three characters with 3x, you’ll find it in here.

Last inserted text register

The read-only register ". contains the text that you last inserted. Don’t make the mistake of using this to repeat an insert operation, though; just tap . for that after you leave insert mode, or have the foresight to prepend a number to your insert operation; for example, 6i.

Filename registers

The read-only register "% contains the name of the current buffer’s file. Similarly, the "# register contains the name of the alternate buffer’s file.

Command registers

The read-only register ": contains the most recently executed : command, such as :w or :help. This is likely only of interest to you if you’re wanting to paste your most recent command into your Vim buffer. For everything else, such as repeating or editing previous commands, you will almost certainly want to use the command window.

Search registers

The read-only register / contains the most recent search pattern; this can be handy for inserting the search pattern on the command line, by pressing Ctrl-R and then / — very useful for performing substitutions using the last search pattern.

Expression register

Here’s the black sheep of the bunch. The expression register = is used to treat the results of arbitrary expressions in register context. What that means in actual real words is that you can use it as a calculator, and the result is returned from the register.

Whenever the expression register is referenced, the cursor is put on the command line to input an expression, such as 2+2, which is ended with a carriage return.

This means in normal mode you can type "=2+2<Enter>p, and 4 will be placed after the cursor; in insert or command mode you can use Ctrl-R then =2+2<Enter> for the same result. If you don’t find this syntax as impossibly awkward as I do, then this may well suit you for quick inline calculations … personally, I’d drop to a shell and bust out bc for this.

Knowing your registers well isn’t as profound a productivity boost as squelching a few of the other Vim anti-patterns, but it can certainly save you some of the annoyance of lost text.

Start Vim with splits

If you want to call Vim to edit more than one file, you can add an option to the command line to open all of the buffers in split windows on startup:

  • -O — Open all buffers in vertically split windows, like :vsp
  • -o — Open all buffers in horizontally split windows, like :sp

Command line splits

The same can also be done for tabs:

  • -p — Open all buffers in tabs, like :tabnew

Command line tabs

Compiling in $HOME

If you don’t have root access on a particular Linux system that you use, or if you don’t want to install anything to the system directories and potentially interfere with others’ work on the machine, one option is to build your favourite tools in your $HOME directory.

This can be useful if there’s some particular piece of software that you really need for whatever reason, particularly on legacy systems that you share with other users or developers. The process can include not just applications, but libraries as well; you can link against a mix of your own libraries and the system’s libraries as you need.

Preparation

In most cases this is actually quite a straightforward process, as long as you’re allowed to use the system’s compiler and any relevant build tools such as autoconf. If the ./configure script for your application allows a --prefix option, this is generally a good sign; you can normally test this with --help:

$ mkdir src
$ cd src
$ wget -q http://fooapp.example.com/fooapp-1.2.3.tar.gz
$ tar -xf fooapp-1.2.3.tar.gz
$ cd fooapp-1.2.3
$ pwd
/home/tom/src/fooapp-1.2.3
$ ./configure --help | grep -- --prefix
  --prefix=PREFIX    install architecture-independent files in PREFIX

Don’t do this if the security policy on your shared machine explicitly disallows compiling programs! However, it’s generally quite safe as you never need root privileges at any stage of the process.

Naturally, this is not a one-size-fits-all process; the build process will vary for different applications, but it’s a workable general approach to the task.

Installing

Configure the application or library with the usual call to ./configure, but use your home directory for the prefix:

$ ./configure --prefix=$HOME

If you want to include headers or link against libraries in your home directory, it may be appropriate to add definitions for CFLAGS and LDFLAGS to refer to those directories:

$ CFLAGS="-I$HOME/include" \
> LDFLAGS="-L$HOME/lib" \
> ./configure --prefix=$HOME

Some configure scripts instead allow you to specify the path to particular libraries. Again, you can generally check this with --help.

$ ./configure --prefix=$HOME --with-foolib=$HOME/lib

You should then be able to install the application with the usual make and make install, needing root privileges for neither:

$ make
$ make install

If successful, this process will insert files into directories like $HOME/bin and $HOME/lib. You can then try to call the application by its full path:

$ $HOME/bin/fooapp -v
fooapp v1.2.3

Environment setup

To make this work smoothly, it’s best to add to a couple of environment variables, probably in your .bashrc file, so that you can use the home-built application transparently.

First of all, if you linked the application against libraries also in your home directory, it will be necessary to add the library directory to LD_LIBRARY_PATH, so that the correct libraries are found and loaded at runtime:

$ /home/tom/bin/fooapp -v
/home/tom/bin/fooapp: error while loading shared libraries: libfoo.so: cannot open shared...
Could not load library foolib
$ export LD_LIBRARY_PATH=$HOME/lib
$ /home/tom/bin/fooapp -v
fooapp v1.2.3

An obvious one is adding the $HOME/bin directory to your $PATH so that you can call the application without typing its path:

$ fooapp -v
-bash: fooapp: command not found
$ export PATH="$HOME/bin:$PATH"
$ fooapp -v
fooapp v1.2.3

Similarly, defining MANPATH so that calls to man will read the manual for your build of the application first is worthwhile. You may find that $MANPATH is empty by default, so you will need to append other manual locations to it. An easy way to do this is by appending the output of the manpath utility:

$ man -k fooapp
$ manpath
/usr/local/man:/usr/local/share/man:/usr/share/man
$ export MANPATH="$HOME/share/man:$(manpath)"
$ man -k fooapp
fooapp (1) - Fooapp, the programmer's foo apper

This done, you should be able to use your private build of the software comfortably, and all without never needing to reach for root.

Caveats

This tends to work best for userspace tools like editors or other interactive command-line apps; it even works for shells. However this is not a typical use case for most applications which expect to be packaged or compiled into /usr/local, so there are no guarantees it will work exactly as expected. I have found that Vim and Tmux work very well like this, even with Tmux linked against a home-compiled instance of libevent, on which it depends.

In particular, if any part of the install process requires root privileges, such as making a setuid binary, then things are likely not to work as expected.

Sync tmux panes

If you have a Tmux window divided into panes, you can use the synchronize-panes window option to send each pane the same keyboard input simultaneously:

Synchronize panes demo

You can do this by switching to the appropriate window, typing your Tmux prefix (commonly Ctrl-B or Ctrl-A) and then a colon to bring up a Tmux command line, and typing:

:setw synchronize-panes

You can optionally add on or off to specify which state you want; otherwise the option is simply toggled. This option is specific to one window, so it won’t change the way your other sessions or windows operate. When you’re done, toggle it off again by repeating the command.

This is an easy way to run interactive commands on multiple machines, perhaps to compare their speed or output, or if they have a similar setup a quick and dirty way to perform the same administrative tasks in parallel. It’s generally better practice to use Capistrano or Puppet for the latter.

Bash history expansion

Setting the Bash option histexpand allows some convenient typing shortcuts using Bash history expansion. The option can be set with either of these:

$ set -H
$ set -o histexpand

It’s likely that this option is already set for all interactive shells, as it’s on by default. The manual, man bash, describes these features as follows:

-H  Enable ! style history substitution. This option is on
    by default when the shell is interactive.

You may have come across this before, perhaps to your annoyance, in the following error message that comes up whenever ! is used in a double-quoted string, or without being escaped with a backslash:

$ echo "Hi, this is Tom!"
bash: !": event not found

If you don’t want the feature and thereby make ! into a normal character, it can be disabled with either of these:

$ set +H
$ set +o histexpand

History expansion is actually a very old feature of shells, having been available in csh before Bash usage became common.

This article is a good followup to Better Bash history, which among other things explains how to include dates and times in history output, as these examples do.

Basic history expansion

Perhaps the best known and most useful of these expansions is using !! to refer to the previous command. This allows repeating commands quickly, perhaps to monitor the progress of a long process, such as disk space being freed while deleting a large file:

$ rm big_file &
[1] 23608
$ du -sh .
3.9G    .
$ !!
du -sh .
3.3G    .

It can also be useful to specify the full filesystem path to programs that aren’t in your $PATH:

$ hdparm
-bash: hdparm: command not found
$ /sbin/!!
/sbin/hdparm

In each case, note that the command itself is printed as expanded, and then run to print the output on the following line.

History by absolute index

However, !! is actually a specific example of a more general form of history expansion. For example, you can supply the history item number of a specific command to repeat it, after looking it up with history:

$ history | grep expand
 3951  2012-08-16 15:58:53  set -o histexpand
$ !3951
set -o histexpand

You needn’t enter the !3951 on a line by itself; it can be included as any part of the command, for example to add a prefix like sudo:

$ sudo !3850

If you include the escape string \! as part of your Bash prompt, you can include the current command number in the prompt before the command, making repeating commands by index a lot easier as long as they’re still visible on the screen.

History by relative index

It’s also possible to refer to commands relative to the current command. To subtitute the second-to-last command, we can type !-2. For example, to check whether truncating a file with sed worked correctly:

$ wc -l bigfile.txt
267 bigfile.txt
$ sed -i '11,$d' bigfile.txt
$ !-2
wc -l bigfile.txt
10 bigfile.txt

This works further back into history, with !-3, !-4, and so on.

Expanding for historical arguments

In each of the above cases, we’re substituting for the whole command line. There are also ways to get specific tokens, or words, from the command if we want that. To get the first argument of a particular command in the history, use the !^ token:

$ touch a.txt b.txt c.txt
$ ls !^
ls a.txt
a.txt

To get the last argument, add !$:

$ touch a.txt b.txt c.txt
$ ls !$
ls c.txt
c.txt

To get all arguments (but not the command itself), use !*:

$ touch a.txt b.txt c.txt
$ ls !*
ls a.txt b.txt c.txt
a.txt  b.txt  c.txt

This last one is particularly handy when performing several operations on a group of files; we could run du and wc over them to get their size and character count, and then perhaps decide to delete them based on the output:

$ du a.txt b.txt c.txt
4164    a.txt
5184    b.txt
8356    c.txt
$ wc !*
wc a.txt b.txt c.txt
16689    94038  4250112 a.txt
20749   117100  5294592 b.txt
33190   188557  8539136 c.txt
70628   399695 18083840 total
$ rm !*
rm a.txt b.txt c.txt

These work not just for the preceding command in history, but also absolute and relative command numbers:

$ history 3
 3989  2012-08-16 16:30:59  wc -l b.txt
 3990  2012-08-16 16:31:05  du -sh c.txt
 3991  2012-08-16 16:31:12  history 3
$ echo !3989^
echo -l
-l
$ echo !3990$
echo c.txt
c.txt
$ echo !-1*
echo c.txt
c.txt

More generally, you can use the syntax !n:w to refer to any specific argument in a history item by number. In this case, the first word, usually a command or builtin, is word 0:

$ history | grep bash
 4073  2012-08-16 20:24:53  man bash
$ !4073:0
man
What manual page do you want?
$ !4073:1
bash

You can even select ranges of words by separating their indices with a hyphen:

$ history | grep apt-get
 3663  2012-08-15 17:01:30  sudo apt-get install gnome
$ !3663:0-1 purge !3663:3
sudo apt-get purge gnome

You can include ^ and $ as start and endpoints for these ranges, too. 3* is a shorthand for 3-$, meaning “all arguments from the third to the last.”

Expanding history by string

You can also refer to a previous command in the history that starts with a specific string with the syntax !string:

$ !echo
echo c.txt
c.txt
$ !history
history 3
 4011  2012-08-16 16:38:28  rm a.txt b.txt c.txt
 4012  2012-08-16 16:42:48  echo c.txt
 4013  2012-08-16 16:42:51  history 3

If you want to match any part of the command line, not just the start, you can use !?string?:

$ !?bash?
man bash

Be careful when using these, if you use them at all. By default it will run the most recent command matching the string immediately, with no prompting, so it might be a problem if it doesn’t match the command you expect.

Checking history expansions before running

If you’re paranoid about this, Bash allows you to audit the command as expanded before you enter it, with the histverify option:

$ shopt -s histverify
$ !rm
$ rm a.txt b.txt c.txt

This option works for any history expansion, and may be a good choice for more cautious administrators. It’s a good thing to add to one’s .bashrc if so.

If you don’t need this set all the time, but you do have reservations at some point about running a history command, you can arrange to print the command without running it by adding a :p suffix:

$ !rm:p
rm important-file

In this instance, the command was expanded, but thankfully not actually run.

Substituting strings in history expansions

To get really in-depth, you can also perform substitutions on arbitrary commands from the history with !!:gs/pattern/replacement/. This is getting pretty baroque even for Bash, but it’s possible you may find it useful at some point:

$ !!:gs/txt/mp3/
rm a.mp3 b.mp3 c.mp3

If you only want to replace the first occurrence, you can omit the g:

$ !!:s/txt/mp3/
rm a.mp3 b.txt c.txt

Stripping leading directories or trailing files

If you want to chop a filename off a long argument to work with the directory, you can do this by adding an :h suffix, kind of like a dirname call in Perl:

$ du -sh /home/tom/work/doc.txt
$ cd !$:h
cd /home/tom/work

To do the opposite, like a basename call in Perl, use :t:

$ ls /home/tom/work/doc.txt
$ DOCUMENT=!$:t
DOCUMENT=doc.txt

Stripping extensions or base names

A bit more esoteric, but still possibly useful; to strip a file’s extension, use :r:

$ vi /home/tom/work/doc.txt
$ STRIPEXT=!$:r
STRIPEXT=/home/tom/work/doc

To do the opposite, to get only the extension, use :e:

$ vi /home/tom/work/doc.txt
$ EXTONLY=!$:e
EXTONLY=.txt

Quoting history

If you’re performing substitution not to execute a command or fragment but to use it as a string, it’s likely you’ll want to quote it. For example, if you’ve just found through experiment and trial and error an ideal ffmpeg command line to accomplish some task, you might want to save it for later use by writing it to a script:

$ ffmpeg -f alsa -ac 2 -i hw:0,0 -f x11grab -r 30 -s 1600x900 \
> -i :0.0+1600,0 -acodec pcm_s16le -vcodec libx264 -preset ultrafast \
> -crf 0 -threads 0 $(date +%Y%m%d%H%M%S).mkv 

To make sure all the escaping is done correctly, you can write the command into the file with the :q modifier:

$ echo '#!/usr/bin/env bash' >ffmpeg.sh
$ echo !ffmpeg:q >>ffmpeg.sh

In this case, this will prevent Bash from executing the command expansion $(date ... ), instead writing it literally to the file as desired. If you build a lot of complex commands interactively that you later write to scripts once completed, this feature is really helpful and saves a lot of cutting and pasting.

Thanks to commenter Mihai Maruseac for pointing out a bug in the examples.

Avoiding the arrow keys

For shell users, moving to the arrow keys on the keyboard is something of an antipattern, moving away from the home row so central to touch typists. It therefore helps to find ways to avoid using the arrow keys in order to maintain your flow.

Bash

The arrow keys in Bash are used to move back and forth on the current command line (left and right), and up and down through the command history (up and down). This leads to the old shell user’s maxim:

“Those that don’t remember history are doomed to press Up repeatedly.”

There are alternatives to both functions. To move left or right by one character on the command line without deleting characters already placed, we can use Ctrl-B and Ctrl-F.

However, to make things a bit faster, there are four other key combinations to move back and forth on a line that are worth learning too:

  • Alt-B — Move back a word
  • Alt-F — Move forward a word
  • Ctrl-A — Move to the start of the line
  • Ctrl-E — Move to the end of the line

Similarly, to move up and down through history, we can use Ctrl-P and Ctrl-N respectively. Don’t forget that rather than keep tapping one of these, you can search backward or forward through history with Ctrl-R and Ctrl-S.

Whoops, I think I just taught you some Emacs.

Vim

To avoid the arrow keys in normal mode in Vim, use h, j, k, and l instead. This can take a little getting used to, but the benefits in terms of comfort for your hands and speed is easily worth it; using the arrow keys is one of the Vim anti-patterns.

If you’re asking “How do I avoid the arrow keys to move in insert mode?”, the answer is that you shouldn’t be moving in insert mode at all; it’s inefficient. When in insert mode you should be inserting text, and maybe using backspace or delete every now and then. To move through text, switching back to normal mode is vastly more efficient.

Moving in command mode is similar. If you need to move around on the command line, in most cases you should pull up the command window so that you can edit the command in normal mode, and then just tap Enter to run it.

In general, when you start thinking about moving through any kind of text, you should reflexively hit Esc or Ctrl-[ to go into normal mode, in order to take advantage of a whole keyboard’s worth of navigation shortcuts.

Vim annoyances

Like any highly interactive application, Vim has a few annoyances even for experienced users. Most of these consist of the editor doing something unexpected, particularly with motions or the behavior of regular expressions and registers. This is often due to vi and Vim being such venerable editors; their defaults sometimes make less sense decades after their creation.

Fortunately, Vim being the configurable editor that it is, many of the more common annoyances can be easily worked around. Bear in mind that not all of these are necessarily problems; if you actually prefer the way something behaves, you should stick with it.

Cursor jumps around while joining lines

If you want to keep the cursor in place when you join lines with J, you can do this, dropping a mark before the operation to which you return afterwards:

nnoremap J mzJ`z

Jumping lands on top or bottom of screen

If you’d like to center the window automatically around the cursor after jumping to a location with motions like n (next search pattern occurrence) or } (end of next paragraph), you can arrange that by remapping to add a zz after the motion:

nnoremap n nzz
nnoremap } }zz

If you don’t need the jump to land you in the exact middle of the screen, but just don’t want it to land right at the edge, you could also use scrolloff:

set scrolloff=10

Note that this also starts scrolling the window with single-line movements like j and k at this boundary, too, which you may not want.

Skipping lines when wrap is set

By default, the j and k keys don’t move by row; they move by line. This means that if you have the wrap option set, you might skip across several rows to reach the same point in an adjacent line.

This can be frustrating if you prefer the more intuitive behavior of moving up to the character immediately above the current one. If you don’t like this behavior, you can fix it by mapping j to gj, and k to gk, which moves by rows rather than lines:

nnoremap j gj
nnoremap k gk

If you think you might need the default behavior at some point, you might want to include reverse mappings so you can move linewise with gj and gk:

nnoremap gj j
nnoremap gk k

Wrapping breaks words

By default, setting wrap displays lines by breaking words if necessary. You can force it to preserve words instead by only breaking at certain characters:

set linebreak

You can define the characters at which Vim should be allowed to break lines with the breakat option. By default this includes spaces and tabs.

Backup files are a nuisance

If you’re developing with a version control system, you might find the in-place backups Vim keeps for saved files with the ~ suffix more annoying than useful. You can turn them off completely with nobackup:

set nobackup

Alternatively, you can set a single directory for them to keep them out of the way with backupdir:

set backupdir=~/.vim/backup

Swap files are a nuisance

Swap files can be similarly annoying, and unnecessary on systems with a lot of memory. If you don’t need them, you can turn them off completely:

set noswapfile

If you do find the swap files useful but want to prevent them cluttering your current directory, you can set a directory for them with directory:

set directory=~/.vim/swap

Accidentally hitting unwanted keys in normal mode

Some of the keys in normal mode bring up functions that aren’t terribly useful to a lot of people, and tend to accidentally get hit when Caps Lock is on, or when aiming for another key. Common nuisance keys are:

  • F1 for help; :help is generally more useful for the experienced user
  • Q to start ex mode; annoying when you intended to start recording a macro with q
  • K to bring up a man page for the word under the cursor; annoying when you’re not writing in C or shell script, and there isn’t a sensible choice of keywordprg for your chosen language

The simplest way to deal with these is to remap them to <nop> so that they don’t do anything:

nnoremap <F1> <nop>
nnoremap Q <nop>
nnoremap K <nop>

Startup message is irritating

If you don’t like the startup message that appears when you open Vim without a filename, you can remove it by adding the I flag to shortmess:

set shortmess+=I

Can’t backspace past start of operation

If you’re in insert mode, by default you can’t use backspace to delete text past the start of the insert operation; that is, you can’t backspace over where you first pressed insert. This is old vi behavior, and if you don’t like it, you can make backspace work everywhere instead:

set backspace=indent,eol,start

Flashing screen is annoying

If you don’t require the error feedback, you can turn off the flashing screen for the “visual bell”:

set visualbell t_vb=

Don’t know what mode I’m in

If you lose track of the current mode, you can get a convenient --INSERT-- indicator at the bottom of the screen with:

set showmode

If you’re having this problem a lot, however, you might want to take stock of how much time you’re spending in insert mode when you’re not actively typing; it’s good Vim practice to stay out of it otherwise, even when you stop to think.

Keep making typos of command commands

If you’re fat-fingering :wq and similar commands a lot due to a sticky shift key, take a look at the Vim command typos fix. Also, don’t forget about ZZ and ZQ as quicker alternatives to :wq and :q!.

Don’t want case sensitive patterns

If you don’t care for case-sensitive searches and substitutions, you can turn it off completely:

set ignorecase

You might find the slightly cleverer behavior of smartcase is better, though. This option only applies case sensitivity if at least one of the letters in the pattern is uppercase; otherwise, case is ignored.

set smartcase

Doesn’t replace all occurrences

The default behavior for the substitute operation, :s/pattern/replacement/, is to replace only the first occurrence of the pattern. To make it replace all occurrences on the appropriate lines, you add the g suffix. If you find that you never need to substitute for only the first occurrence of a pattern in a line, you can add that flag to the patterns by default:

set gdefault

If you do happen to need to replace only the first occurrence one day, you can get the default behavior back by adding the g suffix to your pattern; its meaning is reversed by this option.

Can’t move into blank space in visual block mode

If you need to define a block in visual block mode with bounds outside the actual text (that is, past the end of lines), you can allow this with:

set virtualedit=block

This will let you move around the screen freely while in visual block mode to define your selections. As an example, this can make selecting the contents of the last column in a formatted table much easier.

Filename completion on command line is confusing

If you’re used to the behavior of shell autocomplete functions for completing filenames, you can emulate it in Vim’s command mode:

set wildmode=longest,list

With this set, the first Tab press (or whatever your wildchar is set to) will expand a filename or command in command mode to the longest common string it can, and a second press will display a list of all possible completions above the command line.

Yanking lines is inconsistent

D deletes from the cursor to the end of the line; C changes from the cursor to the end of the line. For some reason, however, Y yanks the entire line, both before and after the cursor. If this inconsistency bothers you, you can fix it with a remapping:

nnoremap Y y$

New splits appear in unintuitive places

If your language of choice is read from left to right, you may find it annoying that by default Vim places new vertical splits to the left of the current pane, and horizontal splits above the current pane. You can fix both:

set splitbelow
set splitright

Caps Lock sends Vim crazy

Caps Lock in normal mode makes Vim go haywire. This isn’t really Vim’s fault; Caps Lock is generally a pretty useless key. It’s often useful for Vim users to remap it, so that it sends the same signal as a more useful key; Control and Escape are common choices. You might even find this reduces strain on your hands.

Several of these fixes were inspired by Steve Losh’s .vimrc file. Thanks also to commenters Alpha Chen and Rob Hoelz for suggestions.

Vim misconceptions

Vim isn’t the best tool for every task, and there’s no reason you shouldn’t stick to your GUI IDE if you know it like the back of your hand and are highly productive in it. The very basic best practices for text editing in general apply just as well to more familiar editing interfaces as they do to Vim, so nobody should be telling you that Vim is right for everyone and everything and that you’re wrong not to use it.

However, because Vim and vi-like editors in general have a lot of trouble shaking off the connotations of their serverside, terminal-only, mouseless past, there are a few persistent objections to even trying Vim that seem to keep cropping up. If you’re someone curious about Vim but you heard it was useless for any of the following reasons, or if you’re an experienced user looking to convince a hesitant neophyte to try Vim, the following list might clear a few things up.

Vim takes too long to learn

If you want analogues to all of the features in your IDE, that would likely take some time, just as it would in any other new editor. However, if all you need to start is to be able to enter text, move around, and load and save files, you just need to know:

  • i to enter insert mode, Esc to leave it
  • Arrow keys to move around in both modes
  • :e <filename> to load a document
  • :w <filename> to save a document
  • :q to quit, :q! to ignore unsaved changes

To do pretty much everything Windows Notepad would let you do, on top of that you’d only need to learn:

  • dd to cut a line
  • yy to copy a line
  • p to paste a line
  • /<pattern> to search for text
  • n to go to the next result
  • :s/pattern/replacement to replace text

From that point, you only get faster as you learn how to do new things. So saying that Vim takes weeks to learn is a bit disingenuous when the essentials can easily be mastered with a few minutes’ practice.

Granted, the arrow keys are a bit of an anti-pattern, but you can worry about that later.

Vim has no GUI

Vim has a GUI version called Gvim for both Windows and Unix. For Mac OS X, the MacVim port is preferred. For experienced users the GUI’s only real advantage over the terminal version is a wider color palette, but it has a toolbar and other GUI elements which some new users may find useful.

Vim doesn’t have spell checking

Vim allows spell checking with system dictionaries, using :set spell and :set spelllang. Misspelt and unknown words are highlighted appropriately.

Vim doesn’t have syntax highlighting

Vim has support for syntax highlighting that can be turned on with :syntax enable, for a very wide variety of programming languages, markup languages, and configuration file syntaxes.

Vim only allows eight colours

This is a limitation of terminal emulators rather than of Vim itself, but most modern GUI terminal emulators allow 256 colours anyway with a little extra setup. The GUI version, Gvim, has full color support with the familiar rrggbb color definitions.

Vim doesn’t have class/function folding

Vim does in fact have support for folding, based on both code structure and manually defined boundaries. See :help folding for details.

Vim doesn’t have autocompletion

Vim allows basic completion using words already in the current buffer, and also more advanced omnicompletion using language dictionaries for constants, variables, classes, and functions.

Vim doesn’t have a file browser

Vim has had the Netrw plugin bundled for some time, which provides a pretty capable filesystem browser.

Vim can’t do network editing

Again, the bundled Netrw plugin handles this. Editing files over FTP and SCP links is pretty transparent. You can open a file on a remote server by entering the following, which will prompt for your username and password:

:e ftp://ftp.example.com/index.html

When you’re done editing, you just enter :w to save the file, and it’s automatically uploaded for you. You can record your FTP credentials in a .netrc file to save having to type in usernames and passwords all the time. URIs with scp:// work the same way; with a good public key infrastructure set up, you can use Vim quite freely as a network-transparent editor.

Vim doesn’t have tabs or windows

The model Vim uses for tabs and windows is rather different from most GUI editors, but both are supported and have been for some time.

Vim has too many modes

It has three major modes: normal mode, insert mode, and command mode. There’s a fourth that’s Vim-specific, called visual mode, for selecting text.

Most editors are modal in at least some respect; when you bring up a dialog box, or enter a prefix key to another command, you’re effectively changing modes. The only real difference is that context shifts in Vim are at first less obvious; the screen doesn’t look too different between normal, insert, and command mode.

The showmode option helps to distinguish between insert and normal mode, a common frustration for beginners. This gets easier when you get into the habit of staying out of insert mode when not actually entering text.

Vim doesn’t work with the mouse

Vim works fine with the mouse, in both Gvim and xterm-like terminal emulators, if you really want it. You can change the position of the cursor, scroll through the document, and select text as normal. Setting the below generally does the trick:

:set mouse=a

However, even a little experience in Vim may show you that you don’t need the mouse as much as you think. Careful use of the keyboard allows much more speed and precision, and it’s quite easy to run a complex editing session without even moving from the keyboard’s “home row”, let alone all the way over to the mouse.

Vim doesn’t do Unicode

Vim supports Unicode encodings with the encoding option. It’s likely you’ll only need to put the below in your .vimrc file and then never really think about encoding in your editor again:

:set encoding=utf-8

Vim isn’t being developed or maintained

The original author of Vim, and its current maintainer and release manager, is Bram Moolenaar. At the time of writing, he is working for Google, and is paid to spend some of his time developing Vim. The development mailing list for Vim is very active, and patches are submitted and applied to the publically accessible Mercurial repository on a regular basis. Vim is not a dead project.

Vim is closed source

Vim isn’t proprietary or closed source, and never has been. It uses its own GPL-compatible license called the Vim license.

The original vi used to be proprietary because Bill Joy based the code on the classic UNIX text editor ed, but its source code has now been released under a BSD-style license.