Start early, compound your productivity

February 27, 2016

When it comes to finances, investing early ensures you get more bang for your buck with compound interest. The same concept can be applied to developer productivity, investing the time to create a productive development environment, will pay dividends for many years down the road. Productivity gains can come in the form of reducing the amount of keystrokes to type common commands, or code blocks, for moving windows around, and lastly for automating your computer setup, so a new laptop can be setup in no time. In this article I’ll mention aliases, snippets, window managers, switching to vim/emacs, and finally storing dotfiles on github.

Like many of you, I heard about these ideas for a long time. I read blog posts about them, watched co-workers swear by vim, or emacs, yet I kept putting off investing the time in my own productivity. This year, I finally decided that one of my new year resolutions will be to set up a productive development environment. I spent most of January learning about those tools, and installing new software. Now i’m on the second month, and I can already say, I feel much more productive in the new setup. Below are some of the investments, that I believe are worth starting with on the path to a more productive environment.

Aliases

Aliases are like shortcuts, defining a shortened version of a long command you tend to repeat often. Think of them like tinyurl for your shell. For example aliasing ‘gcb’ can turn:

$ gcb feature-branch

into:

$ git checkout —branch feature-branch.

And:

$ gcm

into:

$ git checkout master

This is one of the easiest productivity tips to get started. To add an alias, open your ~/.bash_profile in any text editor, and add the following lines:

alias gcb=git checkout --branch
alias gcm=git checkout master

Reload the shell using:

$ source ~/.bash_profile

And try them out!

Snippets

Going up a notch, snippets. When coding, we often keep reusing the same code blocks. Defining a class, defining a function, HTML tags, opening brackets, closing them; all of those can be typed in fewer keystrokes with snippets. How many times have you typed a simple for loop?

for (var i=0; i < n; ++i) {
   // content here
}

Defining a snippet for requires choosing a trigger word, for example ‘fvi’, and depending on the editor you use can be defined in different ways. For example defining the snippet using UtilSnip can look like this:

snippet fvi
for(var i=0; i < $1; ++$i) {
  $2
}
$3
endsnippet

This means when typing fvi, clicking TAB, it will expand to the for loop, and the cursor will be positioned on $1. Pressing TAB again will cycle to the next position in the snippet until $3.

All modern text editors support snippets, read more below: Atom Snippets , Sublime Text, VIM, Emacs,

Window Manager

Minimizing, maximizing, moving to the right, or to the left, are all common operations performed to get running programs positioned in a way that matches the task at hand. Window Managers allow performing those actions using keyboard shortcuts instead of the mouse.

I checked the history of Mac window managers, SizeUp was released in 2009, Slate followed in 2013, while the last one, HammerSpoon was released in 2014. HammerSpoon fits my needs perfectly. It’s free, open source and the configuration is done using a single lua file, stored in the home directory.

Here’s a sample configuration that assigns Cmd+Alt+Ctrl+Left arrow, the action of moving the current window to the left side, and resizing the window to occupy 50% of the screen:

hs.hotkey.bind({"cmd", "alt", "ctrl"}, 'Left', function() hs.window.focusedWindow():moveToUnit(hs.layout.left50) end)

VIM/Emacs

Talking about text editors can easily lead to a flame war between those who prefer IDEs, and those who prefer text editors. Instead i’ll focus on my personal experience using IDEs for a long time (Visual Studio, Sublime, Atom) and switching to VIM earlier this year. And use the following paragraphs to nudge you to consider learning one of those editors, so you can decide what works best for you.

VIM and Emacs are both quite old, dating back to 1976, yet they are still in active development today. The reason for that, is that for purely editing text, vim is far superior to any other IDE. You can find evidence for that in the VIM plugins available for every IDE - VIM Mode for Atom, Vintage mode for Sublime, VsVim for Visual Studio etc.

VIM itself has plugins as well, and with them it’s possible to get VIM and Emacs close to the level of an IDE (auto complete, JIT warnings, refactoring etc.) while keeping the superior editing experience.

Learning VIM is not easy. Gaining mastery of all the commands requires a multi month investment. Also getting VIM customized to your liking will require more tinkering and learning than the other productivity tips in this post. Yet editing text is what we developers do most of the day, so learning VIM has a potential to give the most productivity gains.

Also, with the pre-packaged vim installations below, it’s possible to get started and be productive much faster than starting from scratch. For VIM there’s Macvim that allows running VIM as an app on the Mac, and the SPF13 package by Steve Francia. It gives an experience similar to Atom/Sublime out of the box, with autocomplete and a file explorer visible on the left.

For Emacs, the story is more complicated. Prelude is a classic emacs and comes with popular packages pre-installed. I didn’t get a chance to play with it too much. Just like other editors, there’s a vim mode extension called evil mode. And then there’s Spacemacs — my favorite emacs distribution that attempts to find the middle ground, and provide a solid, unified experience with a great looking UI. Your first experience opening it might be wow. It looks quite amazing (or confusing).

To install vim on the mac:

$ brew install macvim --with-cscope --with-lua --with-luajit --with-python3 --with-override-system-vim

To install emacs on the mac:

$ brew install emacs --HEAD --use-git-head --with-cocoa

Store dotfiles in github

Finally, after you spent some time customizing your environment with aliases, window manager and vim configuration, share all this goodness with others, and exchange ideas by putting your dotfiles on github. Dotfiles are the files beginning with a dot in your home directory that stores configuration files. VIM has .vimrc, Bash has .bash_profile and so on. To get started:

  1. Create a folder called dotfiles somewhere on your computer.
  2. Create a git repository in that folder and upload it to github.
  3. Copy your .bash_profile to the dotfiles folder and symlink it to the home folder:

     cp ~/.bash_profile ~/dotfiles
     rm ~/.bash_profile
     ln -s ~/dotfiles/.bash_profile ~/.bash_profile 
  4. Repeat with other configuration files. Such as Hammerspoon above, vim, emacs etc.
  5. Commit!
  6. Share with your friends and get more tips.

Conclusion

There’s a lot of productivity to be gained by taking an active role in improving your development environment. Spending a few minutes every day to customize your environment, and you’ll be on your way to save more seconds, compounded to minutes, and hours, one step at a time

Implementing Bloom Filter in Javascript

January 16, 2016

Having finished Coursera’s Algorithms course recently, I looked for more interesting algorithms to implement. I found one data structure that can store large amount of data, use less memory than the competition, it’s implementation is quite short and concise, and gives an opportunity to practice some probability theory. That is the Bloom Filter.

You can use a Bloom Filter to answer a question similar to “did I see this item before?”, and you want the answer quickly—usually to avoid an otherwise costly operation to locate the item.

For example, imagine a server whose data is stored across the network. When getting a request, it’s worth checking whether the item even exists before performing a network call to retrieve the item.

Bloom Filter uses a bit array to know which items it has seen before. Each item is saved by having a few bits turned from 0 to 1. The exact bits are determined by running the item through a few hash function and converting the resulting hash number to a number from zero to n where n is the last bit.

The other data structure you might use for this purpose is a hash table. However, because a hash table needs to store the complete key, it uses a lot more memory. If we have 1 billion keys and each key is 255 bytes we will need at least 255GB. That’s more than most EC2 machines will have. However a comparable bloom filter will only need 12GB to store 1 billion keys.

Implementing Bloom Filter in Javascript came with additional surprises. Numbers in Javascript are stored as doubles. All bitwise operations involve converting the 32-bit integer, applying the operation and converting back. As a result, bitwise operations are not very fast. Also, unlike C++, left shift and right shift operations only use the last 5 bits of the operand, which results in: 1 << 128 = 1 In c++ the same operation would have resulted in 0.

My implementation can be found here. It uses murmurhash npm package for hashing the values.

When researching bloom filters, these articles were useful:

Impressions from Algorithms Course

December 28, 2015

I recently finished Princeton’s Algorithms I and II on Coursera. The course covers a lot of material, from data structures (List, Queue, Hash Table, Trees, Graphs, Tries) to algorithms (Sorting, BFS, DFS, Shortest Path, RegEx, Compression).

The course requires more time than other courses. Expect to spend 3 hours a week watching lectures, an hour answering the quiz for every chapter, and the weekly exercises can take a few hours to solve.

Learning the history of CS was also interesting. How David Hilbert, Kurt Godel, Alan Turning, Alonzo Church and John von Neumann asked the basic questions about computing. And that Alan Turing did his PhD in Princeton.

Previously I used to think search problems are related to searching arrays. Now I know it means problems where the answer is a yes/no and that an answer can be checked in polynomial time. Contrast that with other types of problems mentioned in the course: optimization and decision problems.

The exercises are fun. They are done in Java and you will get to learn which algorithms Java uses for sorting / searching / string operations. I was especially impressed with solving a baseball elimination using max flow.

Another very useful data structure is the Interval Tree. Interview questions involving intervals are very popular and this structure makes interacting with intervals much more efficient.

The only topic I wish would have been covered more is dynamic programming. Dynamic programming is a technique of breaking down big calculation into small subproblems, solve the subproblems from the easiest once and use those solutions as part of solving the bigger subproblems.

There’s a Dynamic Programming chapter in this book: Introduction to Algorithms, 3rd Edition. And it’s covered in Stanford’s Algorithms: Design and Analysis, Part 2.

Lastly, the course ended with a song “longest path” recorded by Daniel J. Barrett in 1988 while a student at Johns Hopkins. Unlike the shortest path, there is no known polynomial time algorithm for the longest path. And one can spend their entire graduate school trying to find an algorithm for this problem. Listen to the song here

Happy Learning!

Machine Learning and a new site

April 05, 2015

I just finished taking the Machine Learning class from Coursera. Machine Learning started in the 80s, became more popular in the 90s and with the advent of powerful computers, is now a hot topic. The Machine Learning course came out in 2011, the New Yorker wrote about Machine Learning in 2012 and 2013. By now, everyone is talking about it. This morning I saw A Ted talk by Fei Fei Li about How we’re teaching computers to understand pictures

There are so many excellent online courses available these days. Scott Young took a whole CS degree just by taking MIT courses online. This inspires me to get back to learn Linear Algebra and Calculus. I spent some time researching which courses are available, and compiled them into a list. I also added the list of courses i’m planning to take during 2015.

When taking classes online there’s the question of how to take notes. I started taking notes on paper, until I stumbled upon Nathan Typanski’s blog. He talks about using Latex and Markdown which allow notes to be saved and viewed later. I found that writing formulas in Latex can be time consuming so I’m going back and fourth between taking screenshots of slides and using Latex.

If you are in San Francisco and taking online courses, check out the Colearning: Study Group for Online Courses meetup. Join us if you are around.