Actually using ed

The classic ed editor is a really good example of a sparse, minimal, standard Unix tool that does one thing, and does it well. Because there are so many good screen-oriented editors for Unix, there’s seldom very much call for using ed, unless you’re working on very old or very limited hardware that won’t run anything else.

However, if part of the reason you use vi is because you think it will always be there (it may not be), then you should learn ed too. If your terminal is broken and neither vi nor nano will work, or you break it some other way, your choices may well be between cat and ed. Or even the grand heresy of sed -i

Not a friendly editor

Even more than its uppity grandchild ex/vi, ed has developed a reputation as terse and intimidating to newcomers. When you type ed at the command line, nothing happens, and the only error message presented by default is ?. If you’re reading this, it’s likely your first and only experience with ed went something like this:

$ ed
help
?
h
Invalid command suffix
?
?
^C
?
exit
?
quit
?
^Z
$ killall ed
$ vi

So, ed is not a terribly intuitive editor. However, it’s not nearly as hard to learn as it might seem, especially if you’re a veteran vi user and thereby comfortable with the ex command set. With a little practice, you can actually get rather quick with it; there’s an elegance to its almost brutal simplicity.

It’s also very interesting to learn how ed works and how to use it, not just because it might very well be useful for you one day, but because it occupies an important position in the heritage of the sed stream editor, the ex line editor, the vi visual editor, the grep tool, and many other contexts.

Why is ed so terse?

When ed was developed, the usual method of accessing a Unix system was via a teletype device, on which it wouldn’t have been possible to use a screen-oriented editor like vi. Similarly, modems were slow, and memory was precious; using abbreviated commands and terse error messages made a lot of sense, because the user would otherwise be wasting a lot of time waiting for the terminal to react to commands, and didn’t have a whole lot of memory to throw around for anything besides the buffer of text itself.

Of course, this is almost a non-issue for most Unix-like systems nowadays, so one of the first things we’ll do is make ed a little bit less terse and more user-friendly.

Error messages

Start ed up the usual way:

$ ed

We’ll start by deliberately doing something wrong. Type b and press Enter:

b
?

There’s that tremendously unhelpful ? again. But if you press h, you can see what went wrong:

h
Unknown command

Of course, since it’s the future now, we can spare the terminal cycles to have ed print the error message for us every time. You can set this up by pressing H:

H
b
?
Unknown command

That’s a bit more useful, and should make things easier.

Quitting

You can quit ed with q. Go ahead and do that. If you had unsaved changes in a buffer, you could type Q to quit unconditionally. Repeating yourself works too:

q
?
Warning: buffer modified
q

Command prompt

Let’s invoke ed again, but this time we’ll use the -p option to specify a command prompt:

$ ed -p\*
*

We’ll use that from now on, which will make things clearer both for interpreting this tutorial and for remembering whether we’re in command mode, or entering text. It might even be a good idea to make it into a function:

$ ed() { command ed -p\* "$@" ; }

Basic text input

We’ll start by adding a couple of lines of text to the new buffer. When you start ed with no filename, it starts an empty buffer for you, much like vi does.

Because there are no lines at all at present, press a to start adding some at the editor’s current position:

*a
Start typing a few lines.
Anything you want, really.
Just go right ahead.
When you're done, just type a period by itself.
.

That’s added four new lines to the buffer, and left the editor on line 4. You can tell that’s the case because typing p for print, just by itself, prints the fourth line:

*p
When you're done, just type a period by itself.

A little more useful is n (or pn in some versions), which will show both the line number and the contents of the line:

*n
4       When you're done, just type a period by itself.

So just like in ex, the current line is the default for most commands. You can make this explicit by referring to the current line as .:

*.n
4       When you're done, just type a period by itself.

You can move to a line just by typing its number, which will also print it as a side effect:

*3
Just go right ahead.
*.n
3       Just go right ahead.

Pressing a like you did before will start inserting lines after the current line:

*a
Entering another line.
.
*n
4       Entering another line.

Pressing i will allow you to insert lines before the current line:

*i
A line before the current line.
.
*n
4       A line before the current line.

You can replace a line with c:

*c
I decided I like this line better.
.
*n
4       I decided I like this line better.

You can delete lines with d:

*6d

And join two or more lines together with j:

*1,2j

You can prepend an actual line number to any of these commands to move to that line before running the command on it:

*1c
Here's a replacement line.
.
*1n
1       Here's a replacement line.

For most of these commands, the last line to be changed will become the new current line.

Ranges

You can select the entire buffer with 1,$ or , for short (% works too):

*,p
Here's a replacement line.
Just go right ahead.
I decided I liked this line better.
Entering another line.

Or a limited range of specific lines:

*2,3p
Just go right ahead.
I decided I liked this line better.

These ranges can include a reference to the current line with .:

*2
Just go right ahead.
*.,4p
Just go right ahead.
I decided I liked this line better.
Entering another line.

They can also include relative line numbers, prefixed with + or -:

*2
*-1,+1p
Here's a replacement line.
Just go right ahead.
I decided I liked this line better.

You can drop a mark on a line with k followed by a lowercase letter such as a, and you’re then able to refer to it in ranges as 'a:

*3ka
*'ap
I decided I liked this line better.

Moving and copying

Move a line or range of lines to after a target line with m:

*1,2m$
*,p
I decided I liked this line better.
Entering another line.
Here's a replacement line.
Just go right ahead.

Copy lines to after a target line with t:

*2t4
*,p
I decided I liked this line better.
Entering another line.
Here's a replacement line.
Just go right ahead.
Entering another line.

Regular expressions

You can select lines based on classic regular expressions with the g operator. To print all lines matching the regular expression /re/:

*g/re/p
Here's a replacement line.

(Hmm, where have I seen that command before?)

You can invert the match to work with lines that don’t match the expression with v:

*v/re/p
I decided I liked this line better.
Entering another line.
Just go right ahead.
Entering another line.

Just like numbered line ranges, ranges selected with regular expressions can have other operations applied to them. To move every line containing the expression /re/ to the bottom of the file:

*g/re/m$
*,p
I decided I liked this line better.
Entering another line.
Just go right ahead.
Entering another line.
Here's a replacement line.

Searching

You can move to the next line after the current one matching a regular expression with /. Again, this will print the line’s contents as a side effect.

*/like
I decided I like this line better.

You can search backward with ?:

*?Here
Here's a replacement line.

Substituting

You can substitute for the first occurrence per line of an expression within a range of lines with the s command:

*1s/i/j
I decjded I like this line better.

You can substitute for all the matches on each line by adding the /g suffix:

*1s/i/j/g
*p
I decjded I ljke thjs ljne better.

Reading and writing

You can write the current buffer to a file with w, which will also print the total number of bytes written:

*w ed.txt
129

Having done this once, you can omit the filename for the rest of the session:

*w
129

Like most ed commands, w can be prefixed with a range to write only a subset of lines to the file. This would write lines 1 to 4 to the file ed.txt:

*1,4w ed.txt
102

You can use W to append to a file, rather than replace it. This would write lines 3 to 5 to the end of the file ed.txt:

*3,5W
71

You can read in the contents of another file after the current line (or any other line) with r. Again, this will print the number of bytes read.

*r /etc/hosts
205

The output of a command can be included by prefixing it with !:

*r !ps -e
5571

If you just want to load the contents of a file or the output of a command into the buffer, replacing its contents, use e, or E if you’ve got a modified buffer and don’t mind replacing it:

*E ed.txt
173

If you don’t like seeing the byte counts each time, you can start ed with the -s option for “quiet mode”.

Familiar syntax

Almost all of the above command sets will actually be familiar to vi users who know a little about ex, or Vimscript in Vim. It will also be familiar to those who have used sed at least occasionally.

Once you get good with ed, it’s possible you’ll find yourself using it now and then to make quick edits to files, even on systems where your favourite screen-based editor will load promptly. You could even try using ex.

Update 2016: Recent Hacker News discussion has reminded me of my long-standing mistake in asserting that ed(1) is included in every Unix and Unix-like system’s base installation. This is not even close to true–many others exclude it–and the claim has been removed, which I should have done years ago.

Thanks to A. Pedro Cunha for a couple of fixes to show correct typical output.

Vim as Debian default

The default text editor in installations of Debian and its derivatives is Nano, largely because it’s a simple and small editor. If you’re a Vim user, you might find it a little jarring to be presented with a modeless editor when you run commands like visudo or vipw.

Debian’s alternatives system makes this reasonably easy to adjust. If you have Vim installed, it should be available as one of the possible implementations of the editor alternative. You can check this with the update-alternatives command:

# update-alternatives --list editor
/bin/ed
/usr/bin/nano
/usr/bin/vim.basic

This shows that Vim is available as an alternative with /usr/bin/vim.basic, so you can update the symlink structure that defines the default editor like so:

# update-alternatives --set editor /usr/bin/vim.basic
... using /usr/bin/vim.basic to provide /usr/bin/editor (editor) in manual mode.

Now if you fire up visudo, vipw, or sudo -e you should find that Vim is fired up instead of the editor you didn’t want.

On my own workstation I have the latest Vim compiled from Mercurial and installed into /usr/local via checkinstall, so I had to add this to the alternatives system before I could use it:

# update-alternatives --install /usr/bin/editor editor /usr/local/bin/vim 200 \
    --slave /usr/share/man/man1/editor.1.gz editor.1.gz /usr/local/share/man/man1/vim.1.gz
# update-alternatives --set editor /usr/local/bin/vim
... using /usr/bin/vim.basic to provide /usr/bin/editor (editor) in manual mode.

Other relevant alternatives include the vi implementation for your system, which of course may not necessarily be Vim; some operating systems install the smaller and more vi-faithful nvi.

Bash command existence

If you set environment variables like your EDITOR in your .bashrc file that refer to commands that you expect to be available on the system, it’s prudent to check that an appropriate command actually exists before making the setting.

A common way of approaching this is using which, in a syntax similar to the below, which sets EDITOR to the first executable instance of vi found in PATH, or blank if not found:

EDITOR=$(which vi)

Because the behaviour of which can be unexpected, it’s better practice to use one of Bash’s builtins to do this, either command, type, or hash. I prefer using hash, which searches PATH for a command of the given name, and loads it into Bash’s command hash table if found. An implementation like the below works well:

if hash vi 2>/dev/null; then
    export EDITOR=vi
fi

This ignores any error output from the hash call by redirecting it to the null device, and only defines the value for EDITOR if a matching command is found.

You can compact this syntax into one line:

hash vi 2>/dev/null && export EDITOR=vi

Thanks to commenter Yu-Jie Lin for pointing out the above abbreviation.

Unix as IDE: Editing

This entry is part 3 of 7 in the series Unix as IDE.

The text editor is the core tool for any programmer, which is why choice of editor evokes such tongue-in-cheek zealotry in debate among programmers. Unix is the operating system most strongly linked with two enduring favourites, Emacs and Vi, and their modern versions in GNU Emacs and Vim, two editors with very different editing philosophies but comparable power.

Being a Vim heretic myself, here I’ll discuss the indispensable features of Vim for programming, and in particular the use of shell tools called from within Vim to complement the editor’s built-in functionality. Some of the principles discussed here will be applicable to those using Emacs as well, but probably not for underpowered editors like Nano.

This will be a very general survey, as Vim’s toolset for programmers is enormous, and it’ll still end up being quite long. I’ll focus on the essentials and the things I feel are most helpful, and try to provide links to articles with a more comprehensive treatment of the topic. Don’t forget that Vim’s :help has surprised many people new to the editor with its high quality and usefulness.

Filetype detection

Vim has built-in settings to adjust its behaviour, in particular its syntax highlighting, based on the filetype being loaded, which it happily detects and generally does a good job at doing so. In particular, this allows you to set an indenting style conformant with the way a particular language is usually written. This should be one of the first things in your .vimrc file.

if has("autocmd")
  filetype indent plugin on
endif

Syntax highlighting

Even if you’re just working with a 16-color terminal, just include the following in your .vimrc if you’re not already:

syntax on

The colorschemes with a default 16-color terminal are not pretty largely by necessity, but they do the job, and for most languages syntax definition files are available that work very well. There’s a tremendous array of colorschemes available, and it’s not hard to tweak them to suit or even to write your own. Using a 256-color terminal or gVim will give you more options. Good syntax highlighting files will show you definite syntax errors with a glaring red background.

Line numbering

To turn line numbers on if you use them a lot in your traditional IDE:

set number

You might like to try this as well, if you have at least Vim 7.3 and are keen to try numbering lines relative to the current line rather than absolutely:

set relativenumber

Tags files

Vim works very well with the output from the ctags utility. This allows you to search quickly for all uses of a particular identifier throughout the project, or to navigate straight to the declaration of a variable from one of its uses, regardless of whether it’s in the same file. For large C projects in multiple files this can save huge amounts of otherwise wasted time, and is probably Vim’s best answer to similar features in mainstream IDEs.

You can run :!ctags -R on the root directory of projects in many popular languages to generate a tags file filled with definitions and locations for identifiers throughout your project. Once a tags file for your project is available, you can search for uses of an appropriate tag throughout the project like so:

:tag someClass

The commands :tn and :tp will allow you to iterate through successive uses of the tag elsewhere in the project. The built-in tags functionality for this already covers most of the bases you’ll probably need, but for features such as a tag list window, you could try installing the very popular Taglist plugin. Tim Pope’s Unimpaired plugin also contains a couple of useful relevant mappings.

Calling external programs

Until 2017, there were three major methods of calling external programs during a Vim session:

  • :!<command> — Useful for issuing commands from within a Vim context particularly in cases where you intend to record output in a buffer.
  • :shell — Drop to a shell as a subprocess of Vim. Good for interactive commands.
  • Ctrl-Z — Suspend Vim and issue commands from the shell that called it.

Since 2017, Vim 8.x now includes a :terminal command to bring up a terminal emulator buffer in a window. This seems to work better than previous plugin-based attempts at doing this, such as Conque. For the moment I still strongly recommend using one of the older methods, all of which also work in other vi-type editors.

Lint programs and syntax checkers

Checking syntax or compiling with an external program call (e.g. perl -c, gcc) is one of the calls that’s good to make from within the editor using :! commands. If you were editing a Perl file, you could run this like so:

:!perl -c %

/home/tom/project/test.pl syntax OK

Press Enter or type command to continue

The % symbol is shorthand for the file loaded in the current buffer. Running this prints the output of the command, if any, below the command line. If you wanted to call this check often, you could perhaps map it as a command, or even a key combination in your .vimrc file. In this case, we define a command :PerlLint which can be called from normal mode with \l:

command PerlLint !perl -c %
nnoremap <leader>l :PerlLint<CR>

For a lot of languages there’s an even better way to do this, though, which allows us to capitalise on Vim’s built-in quickfix window. We can do this by setting an appropriate makeprg for the filetype, in this case including a module that provides us with output that Vim can use for its quicklist, and a definition for its two formats:

:set makeprg=perl\ -c\ -MVi::QuickFix\ %
:set errorformat+=%m\ at\ %f\ line\ %l\.
:set errorformat+=%m\ at\ %f\ line\ %l

You may need to install this module first via CPAN, or the Debian package libvi-quickfix-perl. This done, you can type :make after saving the file to check its syntax, and if errors are found, you can open the quicklist window with :copen to inspect the errors, and :cn and :cp to jump to them within the buffer.

Vim quickfix working on a Perl file

Vim quickfix working on a Perl file

This also works for output from gcc, and pretty much any other compiler syntax checker that you might want to use that includes filenames, line numbers, and error strings in its error output. It’s even possible to do this with web-focused languages like PHP, and for tools like JSLint for JavaScript. There’s also an excellent plugin named Syntastic that does something similar.

Reading output from other commands

You can use :r! to call commands and paste their output directly into the buffer with which you’re working. For example, to pull a quick directory listing for the current folder into the buffer, you could type:

:r!ls

This doesn’t just work for commands, of course; you can simply read in other files this way with just :r, like public keys or your own custom boilerplate:

:r ~/.ssh/id_rsa.pub
:r ~/dev/perl/boilerplate/copyright.pl

Filtering output through other commands

You can extend this to actually filter text in the buffer through external commands, perhaps selected by a range or visual mode, and replace it with the command’s output. While Vim’s visual block mode is great for working with columnar data, it’s very often helpful to bust out tools like column, cut, sort, or awk.

For example, you could sort the entire file in reverse by the second column by typing:

:%!sort -k2,2r

You could print only the third column of some selected text where the line matches the pattern /vim/ with:

:'<,'>!awk '/vim/ {print $3}'

You could arrange keywords from lines 1 to 10 in nicely formatted columns like:

:1,10!column -t

Really any kind of text filter or command can be manipulated like this in Vim, a simple interoperability feature that expands what the editor can do by an order of magnitude. It effectively makes the Vim buffer into a text stream, which is a language that all of these classic tools speak.

There is a lot more detail on this in my “Shell from Vi” post.

Built-in alternatives

It’s worth noting that for really common operations like sorting and searching, Vim has built-in methods in :sort and :grep, which can be helpful if you’re stuck using Vim on Windows, but don’t have nearly the adaptability of shell calls.

Diffing

Vim has a diffing mode, vimdiff, which allows you to not only view the differences between different versions of a file, but also to resolve conflicts via a three-way merge and to replace differences to and fro with commands like :diffput and :diffget for ranges of text. You can call vimdiff from the command line directly with at least two files to compare like so:

$ vimdiff file-v1.c file-v2.c
Vim diffing a .vimrc file

Vim diffing a .vimrc file

Version control

You can call version control methods directly from within Vim, which is probably all you need most of the time. It’s useful to remember here that % is always a shortcut for the buffer’s current file:

:!svn status
:!svn add %
:!git commit -a

Recently a clear winner for Git functionality with Vim has come up with Tim Pope’s Fugitive, which I highly recommend to anyone doing Git development with Vim. There’ll be a more comprehensive treatment of version control’s basis and history in Unix in Part 7 of this series.

The difference

Part of the reason Vim is thought of as a toy or relic by a lot of programmers used to GUI-based IDEs is its being seen as just a tool for editing files on servers, rather than a very capable editing component for the shell in its own right. Its own built-in features being so composable with external tools on Unix-friendly systems makes it into a text editing powerhouse that sometimes surprises even experienced users.

Command line editing

By default, the Bash shell uses GNU Readline, which provides a set of Emacs-friendly key bindings that are a pretty workable way to edit long and complicated commands. If you learn a little about the chords available to you in editing Bash commands through this library, and combine that with a little old-school command-line magic, you can work at high speed with decent accuracy quite easily. Alternatively, if you don’t like the Emacs-style keybindings and would prefer to stick to a vi-friendly model, you can use set -o vi to edit your command line directly with vi keybindings.

However, if you’re building a particularly complex string of commands involving a lot of pipes, escapes, and redirections, it often turns out to be handy to actually load them into your favourite editor, to give you full facility to edit them in any way you wish. Bash provides a method for this in the form of its Ctrl+X, Ctrl+E binding.

To use it, you can type anything at the command prompt (including nothing at all) and press Ctrl+X, Ctrl+E to bring up your EDITOR, be it Vim, Emacs, or Nano, with the contents of the command line there to edit. As soon as you save and quit, the command will be run as stated, and will be entered into the command history as if you typed it out on the shell directly.

There’s also a handy built-in Bash shortcut, fc (short for “fix command”) to open the previous command in your editor, allow you to edit it, and then run it automatically when you quit. This is particularly useful if you’ve made a small mistake in a complex line of shell code.

If this happens to bring up the wrong editor, perhaps because your choice doesn’t match that of the system administrator, you can set your personal preference of editor like so:

$ export EDITOR=/usr/bin/vim

You can confirm this is working by checking your environment variables:

$ env

One downside of this method is that without special setup within your editor, you lose some of the benefits of things like tab completion. Fortunately it only takes a little creative mapping to make this work in Vim, taking advantage of the Ctrl+X, Ctrl+F file completion that’s already built in. You could even bind that straight to the Tab key if you don’t otherwise use it.

:inoremap <Tab> <C-X><C-F>