What's in your bash History?

    * listen Speech Icon

Thursday January 4, 2007 11:57PM
by Jonathan Wellons in Technical

AddThis Social Bookmark Button

If you’re like me, you never want to lose a command. I’m constantly searching back through them to find out just what those command line flags were, what the esoteric command is (and where it’s located), and most of all: what in Tcl’s name did I do last month when I installed foobazzulator. First thing to know: control-r.

However, bash ships with a ridiculously short memory: 500 commands. That’s not enough: right now, my powerbook’s iTerm prompt is:

[/Users/jwellons]
111280$

Go into your .bash_profile now and add these lines, before you read on:

HISTFILESIZE=1000000000
HISTSIZE=1000000

Unless you’re very precocious, that should square you away for about 10 years (I’m at about 22 months since enlightenment).

Once you develop control-r as a fast way to search your command history, you’ll notice immediate stress relief, and as I do everyday, wish you’d done this years before.

Now that you have commands piling up, you can do more than just use them. You can analyze them too, for fun and profit (since time = money)!

The basic starter is

cut -f1 -d" " .bash_history | sort | uniq -c | sort -nr | head -n 30

In other words, just what do you spend so much time typing? Here’s the money: Anything in at least the top ten, assuming it is not from some massive one-time splurge, needs to be aliased to a one or two letter shorthand.

My top command is v for vim, which I’ve executed 9225 times. If those two extra letters, which require me to switch hands, then use adjacent fingers take an extra half-second each time: that’s well over an hour wasted! And what a terribly drudgerous and unhealthy hour it is too: mindlessly typing im 9225 times!

Even worse, who knows how desperately I needed to open that file, and yet I needed to type superfluous characters, probably messing them up on the first three tries.

Actually, that’s only my second highest command. cl tops the list at 11116. You see, last time I did this analysis, I noticed I was spending (yes, spent, like lines of code, there is no glory in using a lot of them) many commands first cd‘ing to a directory, then listing the contents. Here’s cl:

# Compress the cd, ls -l series of commands.
alias lc="cl"
function cl () {
   if [ $# = 0 ]; then
      cd && ll
   else
      cd "$*" && ll
   fi
}

There you have it, accompanied by its most common typo. Add this to your bash profile now, and you’ll add several hours to your life. Hurry! Don’t wait! It may not seem like much now, but no one lies on her deathbed wishing she had spent more time first changing directories, waiting for it to return, then typing ls.

Since I know you’re dying to see the rest, here’s my whole thirty, unaliased for readability:

11116 cl
9225 vim
7833 ll The alias of ll depends on the OS I’m on, but for my Mac, it’s ls -AGlFT.
5145 cd
4858 clear
4563 rm
3950 alias to log into the master db server at my old workplace
3740 lt version of ll that puts most recently modified stuff at the bottom. Mandatory! Add -tr to ll above
3435 mod a function that does lt, but runs it through tail
3236 mv
2103 ls
1887 grep
1863 perl
1834 df
1767 mzscheme
1679 alias to log into my personal server
1580 g++
1544 cat
1186 scp
1080 find
988 man
925 echo
921 mkdir
907 sudo
888 history
841 alias to log into the dev server at my old workplace
831 cp
763 gpg
758 locate
733 gzip

Depending on response, we may do another section with pie charts, finding all the typo permutations for simple commands (there’s a lot), and searching for common sequences.

What I really want to see, though, is your top commands.

Comments

9948 vim
4999 make
4501 ls
2772 cd
2202 grep
1980 man
1934 ssh
1440 bc
1224 cat
1124 fg
Anonymous | January 5, 2007 01:04 AM

Great post!

It prompted me to put something in my .aliases file which I've always been meaning to add:

# mkdir & cd into it
function mc() {
  mkdir -p "$*" && cd "$*" && pwd
}

(Don't name it mc if you have Midnight Commander installed. I haven't so it's OK)
I also use:



HISTCONTROL=ignorespace

so I can prefix a command with space and it won't get added to the history file. The only drawback there is that it also isn't in the previous command list you get when you press the up arrow key.

Here's some others I use a lot:

alias home="ssh me@server.at.my.house"
# make top default to ordering by CPU usage:
alias top='top -ocpu'
# more lightweight version of top which doesn't use so much CPU itself:
alias ttop='top -ocpu -R -F -s 2 -n30'

this general rule is quite useful for servers you commonly connect to:

alias servername="ssh servername"

...especially if you also set up ssh correctly so that you automatically log in without a password prompt.
Jon Evans | January 5, 2007 03:52 AM

2490 cd
2339 perl
1645 ll  [alias for 'ls -l']
1333 ls
1043 man
 796 bbedit
 708 perldoc
 687 make
 603 tree
 484 locate

Anonymous | January 5, 2007 05:29 AM

My history was at the default size until three minutes ago, but it's informative nonetheless:

101 ls
70 gvim
61 cd
38 svn
30 alias to cd to a particular directory
27 fs [alias for "find . -wholename '*.svn' -prune -o -type f -print0 | xargs -0 grep -li"]
16 alias to cd to a particular directory
15 alias to cd to a particular directory
14 clear
13 tail

My most-used aliases, other than directory shortcuts, are fs (for quick string searching), rez (#11, restarts Zope), and gdiff (#12), which is an alias for "svn diff > .svn.tmp; gvim .svn.tmp" -- good quick way to go over diffs before a commit.
Ian McCracken | January 5, 2007 05:47 AM

I'd recommend a more moderate number. I once had it set high like that, but after awhile I noticed that bash took longer and longer to start up, upwards of 5-10 seconds. It took quite awhile to figure out that it was my bash history file.
Simon Ditner | January 5, 2007 06:30 AM

I'm using a debian based system & cannot find my .bash_profile file
to try the stuff quoted..
Can you help me as to where, I can modify my entires for histfilesize etc ?

thx
easylife | January 5, 2007 07:43 AM

Sweet post, I'm going to jack up my history allotment. Small correction - where you give the shortcut to see your top commands I think you meant:

cut -f1 -d" " .bash_history | sort | uniq -c | sort -nr | head -n 30

rather than:

cut -f1 -d" " .bash_profile | sort | uniq -c | sort -nr | head -n 30

Kevin | January 5, 2007 08:22 AM

Actually here are a few things. I personally use
HISTCONTROL=ignoredups in my bashrc, since I use up/down arrows a lot to go back and forward in the command history. And if I just typed ls -lhS a few hundred times to see how far did my download progressed (yes, I know there is a watch command and I do use it after first 2-3 thousand iterations :) ) I don't want to wade through them again. But still keeping the history forever is a "Good Thing". As already mentioned in your first top 10 though you need to change .bash_profile to .bash_history. CTRL-R is great, I would hit it by accident once in a while and never figured what that does. Here is my list
160 cd
151 ls
54 vim
47 svn
35 sudo
34 rm
30 for
24 su
23 cat
20 scp
Dmitriy Kropivnitskiy | January 5, 2007 10:12 AM

Hee, hee... I love geeked out stuff like this. Thanks for the tip. I didn't know about ctrl-r (I'm your stereotypical self-taught, work alone, need to get out more developer and I often find basic things like this that I just never stumbled across before.)

ls
cd
grep
sudo
php
man
web <-- an alias I use to cd into my web root
locate
whois
ls <-- I have this aliased to: ls -FahGsilt
Stuart | January 5, 2007 10:27 AM

Dear Kevin and Dmitriy Kropivnitskiy,

Ouch! Good Call. It should be fixed now.

Jonathan
Jonathan Wellons | January 5, 2007 10:33 AM

Dear easylife,

I don't know offhand, but do you have a .bashrc file instead? It could also be that you simply have to create the file for it to be used. Touch .bash_profile in your home directory and put something simple (echo Bonjour) in it to see if it's being read.

Jonathan
Jonathan Wellons | January 5, 2007 10:38 AM

Dear Simon Ditner,

Fair enough. What number do you use now?

Jonathan
Jonathan Wellons | January 5, 2007 10:39 AM

C-r is great, but these two bindings are even better:

# .inputrc
# "\e[A" and "\e[B" being whatever your terminal uses for up & down.
"\e[A": history-search-backward
"\e[B": history-search-forward

This way, if you type in "vim " and push up, you cycle through all your previous commands starting with "vim ". With no text inserted, it cycles through history just like the default bindings do.

I'm switching to zsh at the moment, so each history file is just full of alias tests. The first thing I did was getting those two bindings working in zsh exactly like they do in bash.
Leland | January 5, 2007 11:05 AM

This is a great idea. However, I can easily have half a dozen bash screens open at once (mutt, ssh to various machines, gentoo emerge, dev, etc). While the histories work fine while in each session. The last shell exited determines which series of commands gets saved to the .bash_history file. Most of my history gets lost.
shellgame | January 5, 2007 11:21 AM

Dear shellgame,

Check out the histappend shell option on the bash man page.

Jonathan
Jonathan Wellons | January 5, 2007 12:44 PM

Jonathon,

Just what I needed, thanks!
shellgame | January 5, 2007 02:53 PM

Nicely written article, great comments - thanks! Especially the history-search-backward/forward config was new to me.

My current favourite commands seem to be `svn` and `see` (opens SubEthaEdit).
Tommi | January 5, 2007 03:11 PM

You can save five keypresses by using ^L instead of 'clear' (well, 4 1/2 if you insist on counting the control key).
Keith | January 8, 2007 03:36 PM

Nice post. Great comments!

Since we're sharing the guts of our .bash* files, here's one of my favorites:

lsl () {
    ls --color=always -C $* | less -r
}

ls with color that doesn't scroll of the screen!
Marc Mims | January 8, 2007 07:48 PM

Hi Jonathan,

A couple of comments. Firstly, your sys. admin. may not be too impressed with a huge file appearing in their incremental backups that's almost guaranteed to change and therefore appear each day. At least if their backup technology backs up the whole file, e.g. tar. :-)

Second, you cl() definition seems overly complex. Why not

cl() { cd "$@" && l; }

"$@" is preferable to "$*" since it preserves quoting better. Both disappear if there are no arguments resulting in a plain "cd" so no need to test $#. And since ll-functionality is so common I've always wondered why people make it the same character twice in a row since this takes longer to type that to separate characters. Personaly, I use `l' as `ls -l'.

And apart from things that must be shell built-ins, e.g. those involving cd, I always put `l', `v', `ltr', etc., in ~/bin, often with an exec, so they work at the ! prompt of whatever program I'm in which may not kick off bash.

Cheers,

Ralph.
Ralph Corderoy | January 9, 2007 04:33 AM

I just learned about the bash history file size, but here's mine as of today:
210 sudo
25 cd
21 less
20 ls
18 nano
16 su
15 ssh
12 man
9 rm
7 wget
Paul | January 9, 2007 08:02 AM

I'm not going to declare this post a bad thing, but there's one thing everyone should be aware of - if you alias the hell out of your environment, you're going to be lost when you walk over to a co-worker's (spouse's, friend's, etc.) desk. (You'll figure it out, but you can't concentrate on the important problems if you're habitually typing commands that don't work.) Saved keystrokes do not necessarily equal saved time.

It can trip you up in your own environment, too - what if you "cl" into directories with hundreds of files? Do you really want to fill your screen each time? "cd" and "ls" are separate for a reason.

I'm not saying don't do this, but be aware of the power (and drawbacks) of habituation.
Jay McGavren | January 9, 2007 09:51 AM

Dear Ralph Corderoy,

I'm my own sys admin now, but back when I wasn't, our systems were moving Gigs of data per day to offsite storage. I'm sure my .bash_profile was not a significant burden. If this is an issue, there's got to be some sort of compromise the sysadmin would agree to. Even 10,000 commands is small compared to many files.

I incorporated your cl simplification, and I'm giving l a try. Thanks!

When you say you put the aliases in ~/bin, are you referring symlinks? If so, that's something I never thought of.

~Jonathan
Jonathan Wellons | January 9, 2007 06:40 PM

Dear Jay McGavren,

cd certainly appears in my list. You'll notice I used it almost half as much as I used cl. There's plenty of times when, as you suggest, I know exactly what I'm planning to do in a directory and can go directly to executing, editing, etc.

Nevertheless, I'd be remiss not to admit there's been a couple times when I've cl'ed to a directory and had to wait for 2000 files to scroll past. Can't win 'em all...

~Jonathan
Jonathan Wellons | January 9, 2007 06:45 PM

Dear Keith,

I'm currently using c for clear. I'm starting to try ^L instead.

~Jonathan
Jonathan Wellons | January 9, 2007 06:46 PM

When it comes to saving typing there is one favorite you just cannot passover:

tab

...as in tab-completion. It hails from, nay is practically an alias for, BASH Completion. While it's not handy in completing short commands, it is indispensable in completing commands you can't quite remember the spelling of. AND it's good for completing directory and filenames when using them as arguments.
Bill | January 10, 2007 06:39 AM

I have a few boxes kicking around, so I did my top 10 list of commands from a Free BSD machine that I operate at work for the phone company.
I used ' history | cut -f 3 > usr_hist ' &&
and then ' cat usr_hist | cut -f 1 -d " " | sort -u ' to generate this list, shorter than the entire great gobs of listed commands.
My top commands turn out to be :
apropos
cat
cd
chmod
cp
exit
history - I use this a lot
ircd - local script
ls
man
pwd
rehash
scp
sort
su
ssh
top
vim
wc
which

I'm sure it differs slightly from machine to machine, but in a nutshell there's *nix. I really like the personal touches that others have shared here already, I've picked up a few nuggetts.

thanks,
Craig
Craig Sinclair | January 10, 2007 12:06 PM

Well, one thing I should note that from what I found out increasing the maximal bash history size also increases the memory footprint of each bash instance. Back when I had only 512 MB of RAM, and tried a history size of 100,000 lines, each bash occupied several percents of my RAM. So I ended up decreasing it to 25,000 lines.

Recently, I found out that on my Mandriva 2007 system, sometimes the .bash_history gets reset and contains less than ten commands instead of the 25,000 commands that existed previously. To combat it, I implemented an hourly Subversion-based backup of the .bash_history today. It's a workaround instead of a solution, but at least I would be able to revert to a good .bash_history file.
Shlomi Fish | January 13, 2007 06:39 AM

Dear Shlomi Fish,

Thanks for the good point about backups. Like anything valuable, your commands should be kept safe.

I have a special command that I use that copies each of these files off to a backup directory:

   .bash_history
   .mysql_history
   .psql_history
   .cpan/histfile

~Jonathan
Jonathan Wellons | January 13, 2007 02:45 PM

Some nice tips here. A couple of other things that I use a lot:

- Meta-. which cycles through previously used arguments.
- A bash function called recd which makes it really simple to cd to often-used directories.
Marc | January 15, 2007 09:55 PM

Thanks, the lc is realy helpfull and I don't quite understand, why I didn't had thought of this. :)
Hartmut | January 21, 2007 09:45 AM

Here's mine, at the default size. It's mostly work-related, and it's probably pretty obvious what I do at work:

235 perl
72 cd
42 ls
29 clear
23 mv
19 mkdir
15 ping
13 svn
10 top
8 sort
7 kill
6 man
5 rm
4 wget
2 ssh
2 diff
1 zip
1 sudo
1 less
1 ipconfig
1 gzip
1 ;s
1 16^16
1 15^16
Lindsey Kuper | January 28, 2007 12:28 PM

...and then I started making a make.sh for every project on my work machine, and I haven't had to type perl since:

240 svn
84 ./make.sh
79 cd
49 clear
17 ls
12 rm
5 sudo
5 man
5 find
1 top
1 mkdir
1 lines
1 ./ma
Lindsey Kuper | May 11, 2007 05:27 PM

Dear Lindsey,

Interesting how svn jumped up so high. Assuming svn didn't change the distribution of commands, it almost looks like make.sh replaced a disproportionate number of perl's (about half perl to about one-third make.sh). More data is surely needed, but could that be because make.sh more efficient or less error-prone? What does your make.sh do?

~Jonathan
Jonathan Wellons | May 11, 2007 08:02 PM

Looks like my biggest make.sh scripts replace five commands: three perl, one zip, and one ruby. Not all of them do so much, though. Some only replace two commands.

It makes sense that svn has jumped up. I have a lot more stuff in Subversion than I did in January, and I'm working with another developer now, which means I check in more often.
Lindsey | May 16, 2007 01:26 AM

ctrl-r:
If you don't actually want to execute the command right away and would like a chance to edit it, press right arrow.

Passwords:
If you ever accidentally type a password at the prompt, be sure to cleanup your history file!
kael | July 16, 2007 02:46 PM

Thanks Kael, that's a good point about passwords.
Jonathan Wellons | July 16, 2007 06:06 PM

# tcsh aliases
alias mcd 'mkdir -p \!*; cd \!*'
alias rcd 'setenv OLD_DIR `pwd`;cd ..;echo $OLD_DIR;rd "$OLD_DIR"; unsetenv OLD_DIR'
Anonymous | July 17, 2007 04:45 AM

Dear Anonymous,

I don't have rd on my system. Is that a typo?
Jonathan Wellons | July 17, 2007 11:28 PM

Wow,

This was a nice, juicy, bite-sized morsel of info.
I put this in my bash profile for another website I do.
It used csh as the default, so right away I was behind the 8 ball...

Thanks for the great tips!!
Al U | March 20, 2008 10:34 AM

Dear Al U,

Thanks for letting me know you were able to use it!

Best,
Jonathan
Jonathan Wellons | March 20, 2008 11:15 AM

Jefferson Wang has found out how to format and add timestamps to your bash history logs. Note that this is supported only in bash 3, as far as I know. Everything below the line is his writing:

======================

By adding a export value
HISTTIMEFORMAT="[ %Y-%m-%d %H:%M:%S ] "
in my /etc/env.d/00basic does the magic~
( the results are like this
80 [ 2008-03-23 23:22:38 ] cd /
81 [ 2008-03-23 23:22:38 ] ls
82 [ 2008-03-23 23:22:40 ] history
83 [ 2008-03-23 23:22:47 ] clear
84 [ 2008-03-23 23:22:55 ] vi /etc/env.d/00basic
85 [ 2008-03-23 23:23:12 ] env-update && source /etc/profile
86 [ 2008-03-23 23:23:17 ] ls
87 [ 2008-03-23 23:23:19 ] history
)
(not necessarily /etc/env.d/00basic, anywhere to export the value
is ok,
like /etc/profile, ~/.bash_rc, etc.)
Jonathan Wellons | March 25, 2008 12:02 PM

I guess about so:

la (alias la="ls -sahl --color=auto")
cd
man
cat
grep
vi
reboot
vim
snmpwalk
Barryke | April 17, 2008 03:38 PM

I don't see a lot of fg or %N commands -- aren't people using ^Z to suspend programs?

Here's my work commands:

112 cd
107 fg
95 svn
92 lt [ls -lrt]
83 j [jobs]
66 vi
48 ./[script I was working on recently].pl
31 ls
21 perl
21 cat
20 hm [cd's to the source code root]
17 w
16 grep
15 pwd
15 ps
14 %3
12 jn [in my personal shell account, this prints out new
mail and newsgroup info; for work it's undefined :)]
10 perldoc
10 bin
9 [another script].pl
8 less
7 srd [screen -rd]
7 ping
7 %2
6 rm
6 exit
6 comp [cd /blah/blah/comp]
6 %4
6 [cleans the palate]
5 man
steveo | May 21, 2008 11:43 AM

Quite a few good tips have I found here - thanks!

Steveo - I guess the program called screen has almost completely dislodged the need for fg, jobs and the like. I even have my screen sessions reattached (or created if one is missing) automatically from my .bashrc.

Personally I don't have many aliases, as I tend not to personalize my environment too much, I prefer using the plenty of Bash shortcuts already available on every box.

As to combining cd and ls, it becomes useless, when you utilize tab-completion extensively, which combined with bashcompletion scripts is even more inteligent, because it will complete your cd command with directories only, leaving nondirectories aside - very handy. Not to mention lots of other inteligent completions, that bashcompletion provides.

But to the history... I personally use C-r a lot, and after having recalled some longer command I use Ctrl+Left/Right to skip through (or C-w to delete) space-separated words and Alt+Left/Right to skip through (or Alt+Backspace to delete) whatever-separated words. That's very usefull whenever you need to change some argument in the middle of a long command (like a loop with a few commands inside). You can even get near the right place in your previous long command almost instantly by typing C-r and then a unique fragment of the previous command that needs to be altered (like a variable name or unique value in a aparam) - Bash will place the the cursor exactly there. Pressing C-r several times will cycle backwards through all the matching fragments if more than one matches in the same history line.

Another tip loosely connected with Bash history is using "cd -" if you want to toggle between two directories, and using $OLDPWD builtin variable to access something from a directory you have just been to, like `cd ~/Pictures/; mv $OLDPWD/pr0n.jpg .`

Greets!
Emil Wojak | May 29, 2008 09:37 AM

And one more thing about clearing your accidentally typed passwords from history. Clearing your whole history, although easly acomplishable (like with `export HISTFILESIZE=0 then C-d` or history -c), is something we definetely do NOT want, do we?
So history -d N comes in handy, where N is the ID of the command in history, which you can learn by issuing the standalone history command or with the number of last commands to show as a single argument.
Emil wojak | May 29, 2008 09:56 AM 
