Speeding things up with tmpfs

Posted on October 3, 2011, 12:53 am, by Mayank Jain (makuchaku) and received No Comments ».

Since I have 6GB of RAM in my new laptop, I thought (while watching the first episode of Big Boss 5) that lets put it to some good use.

Did some research & found out that I can actually create a small tmpfs partition from a portion of my extra RAM and use it to super-charge my disk access.

What if, I could keep the most accessed parts of my hard disk on this partition & somehow figure out how to persistently do this, I think I can really really speed up the disk access.

I did a quick test to confirm this (/run is mounted on my Ubuntu 11.10 Beta2 as tmpfs partition)

Test – time creation of a 1GB dummy file.

On my regular 7200 rpm disk

On the /run partition (which is mounted as tmpfs)

This clearly shows the benefit of using tmpfs for disk access.

In the posts following this post, I would try to document my experiences with tmpfs.

AppDok is here!

Posted on August 10, 2011, 12:06 am, by Mayank Jain (makuchaku) and received No Comments ».

Tomorrow is when AppDok’s might is going to be tested!

Wishing good luck to the AppDok‘s launch tomorrow :)

Fingers crossed!

Tags: ,

Trip notes – Gangtok, Sikkim

Posted on July 16, 2011, 10:11 pm, by Mayank Jain (makuchaku) and received 3 Comments ».

With this post, I am trying to start a new habit – of documenting tidbits about my trips with my wife (Kanika) to places worth remembering & sharing about.

I hope, with this, I will also renew the long lost will to sit and write for my dear blog :)

Gangtok

Some quick facts about our journey…

  1. Our trip was scheduled from 29th June 2011 to 3rd July 2011.
  2. Flight tickets costed us about INR 13,000 (we had booked our tickets about 1 month in advance)
  3. Our stay was at a local guest house in Gangtok (arranged by Kanika’s mom) – cost – INR 100 /night
  4. Travel from Bagdogra airport to Gangtok (guest house drop) – INR 820 (shared cab with another couple) (Maruti Van)
  5. Travel from Gangtok (Guest house pickup) to Bagdogra Airport – INR 1,500 (same Maruti van, same driver, not shared)
Notes about Gangtok…
Things which you should know before planning a trip there…
  • Rains are a complete badass in Gangtok. It used to rain for complete nights during our stay (and during the days as well). Roads leading to the hills which are not well built get dangerously muddy & almost unusable.
  • Paying extra to get an experienced driver and a good (& heavy) car is a decision you should always try to be in favor of.
  • Season to visit – Oct/Nov and March/April (costly) – Its the best time to visit as the badass rains have either not started yet or just stopped. Winter is either setting in or giving way to summers. If lucky, you can also get to see snowfall at this time.
  • Off season – June/July (cheaper) – Though we had no choice about our travel dates – we did enjoy the climate at this time of the year. Not to mention, visiting Gangtok during this part of the year has its own advantages :)
  • Travel inside Gangtok is VERY costly. The private taxi driving business is soo booming here that almost every next youngster is a private taxi driver in Gangtok. Be ready to shell out $$ if you plan to take up private cabs for your sigh seeing. What costed us just INR 820 one way (luckily we found a couple to share the cab with) – got inflated to INR 1500 just because we were the only folks travelling to the Airport that morning from our route.
  • There is NO airport in Gangtok. The only airport near Gangtok is in Bagdogra (West Bengal) – which is about 135 kilometers from Gangtok. This travel can take anywhere from 3 to 6 hours based on the condition of roads (& traffic jams). Start early if your flight is in the first half of the day.
  • People of Gangtok are amazingly simple and friendly. I have not seen a city this friendly. Even Pune isn’t any close to Gangtok that way. Kudos to them :)
  • When traveling first time to Gangtok, try to locate your destination hotel/lodge in reference to the MG Market. Its the heart of the city, has 3 large taxi stands built around it. Also, every driver knows this place. If at all possible, get a guest house near this market. Ours was about 10 kms further uphill and we really did witnessed some interesting times – trying to figure out how to reach our guest house.
  • If you plan to visit Nathula Pass, carry two photographs of yours. Not required, but also carry some identification papers (xerox) – as any visit to the Nathula Pass is sanctioned by the Army.
In our 3 days of stay at Gangtok, I think we saw almost all of what was there to see in this season.
  1. 1st Day – We booked a cab for INR 1000 which took us to the North & central tour of the city. It was a decently fulfilling day.
  2. 2nd Day – We opted to rest. Visited the MG Market, bought souvenirs, saw Delhi Belly at a local theater :) We also booked our travel to Nathula Pass (INR 600 per person, shared cab)
  3. 3rd Day – The journey to Nathula Pass was awesome. The valley below you looks breathtaking when you are going uphill. And at 15,000 feet, the climate is awesomely chilly. Hats off to the Army Jawans who live in such a harsh climate.

Gist
Its an amazingly breathtaking place. If you are willing to travel on long road journeys & enjoy the journey more than the destination, its THE place for you.

I just wish, there was an airport inside Gangtok (or atleast Sikkim).

To end this post, I would like to show a brilliant piece of art I saw at the Handicrafts museum. I am completely in awe of the symmetry this drawing was made with. Whats even more brilliant, was the way this symmetry was achieved.

Lord Gautama Buddha

Date arithmetic without having to worry about impact on dates, months or years!

Posted on May 9, 2011, 4:16 pm, by Mayank Jain (makuchaku) and received No Comments ».

Last night, my wife asked a very simple question… - ”In shell scripting, given a date, how can I subtract 5 hours from it?”

The question was particularly interesting because the moment you think about any arithmetic on dates, you need to calculate the impact on all the other values. For example, subtracting 5 hours from 0200 hours would shift the date by -1. Similar for months, years & what not. Things get even more complicated by leap years, etc.

So, back to question. How do I subtract 5 hours from a given date? Usage of any other programming language is not allowed (Perl or Ruby).

I tried, searched internet for some references – I did came across some candid solutions, but they either resorted to Perl, or were full fledged shell scripts themselves.

And then it hit me.

If I convert the given timestamp into seconds since Epoch, subtract the number of seconds in 5 hours & convert the number of seconds back to a timestamp – it’ll be a whhakeroonie!!!

Hence, this…

D="20110406164224"
date -d "${D:0:8} ${D:8:2}:${D:10:2}:${D:12:2}"
==> Wed Apr  6 16:42:24 IST 2011
date -d "${D:0:8} ${D:8:2}:${D:10:2}:${D:12:2}" +"%s"
==> 1302088344
expr 1302088344 - 18000
==> 1302070344
date -d @1302070344
==> Wed Apr  6 11:42:24 IST 2011

As you can see, the result is -5 hours from the timestamp.

Simple MapReduce with Javascript

Posted on May 8, 2011, 12:10 pm, by Mayank Jain (makuchaku) and received 2 Comments ».

The best way to understand MapReduce is to actually see & feel it in action. Following is the simplest possible example I could cook… (Gist link)

  1.  
  2. var Job = {
  3.  
  4.   data : [
  5.           "We are glad to see you here. This site is dedicated to",
  6.           "poetry and to the people who make poetry possible",
  7.           "poets and their readers. FamousPoetsAndPoems.com is",
  8.           "a free poetry site. On our site you can find a large",
  9.           "collection of poems and quotes from over 631 poets",
  10.           "Read and Enjoy Poetry",
  11.           "I, too, sing America",
  12.           "I am the darker brother",
  13.           "They send me to eat in the kitchen",
  14.           "When company comes",
  15.           "But I laugh",
  16.           "And eat well",
  17.           "And grow strong",
  18.           "Tomorrow",
  19.           "Ill be at the table",
  20.           "When company comes",
  21.           "Nobodyll dare",
  22.           "Say to me",
  23.           "Eat in the kitchen",
  24.           "Then",
  25.           "Besides",
  26.           "Theyll see how beautiful I am",
  27.           "And be ashamed",
  28.           "I, too, am America"
  29.         ],
  30.        
  31.  
  32.   map : function(line) {
  33.     var splits = line.split(" ");
  34.     var temp = [];
  35.     for(var i=0; i<splits .length; i++)
  36.     {
  37.       temp.push({key : splits[i], value : 1});
  38.     }
  39.     return temp;
  40.   },
  41.  
  42.  
  43.   reduce : function(allSteps) {
  44.     var result = {};
  45.     for(var i=0; i<allSteps.length; i++)
  46.     {
  47.       var step = allSteps[i];
  48.       result[step.key] = result[step.key] ? (result[step.key] + 1) : 1;
  49.     }
  50.     return result;
  51.   },
  52.  
  53.  
  54.   init : function() {
  55.     var allSteps = [];
  56.     for(var i=0; i<Job.data.length; i++)
  57.       allSteps = allSteps.concat(Job.map(Job.data[i]));
  58.      
  59.     var result = Job.reduce(allSteps)
  60.     console.log(JSON.stringify(result));
  61.   }
  62.  
  63. }; // Job
  64.  
  65.  
  66. Job.init();
  67.  

Copy & paste this code in your browser’s JS console.

Console output…

  1.  
  2. {"631":1,"We":1,"are":1,"glad":1,"to":5,"see":2,"you":2,"here.":1,"This":1,"site":2,"is":2,"dedicated":1,"poetry":3,"and":4,"the":5,"people":1,"who":1,"make":1,"possible":1,"poets":2,"their":1,"readers.":1,"FamousPoetsAndPoems.com":1,"a":2,"free":1,"site.":1,"On":1,"our":1,"can":1,"find":1,"large":1,"collection":1,"of":1,"poems":1,"quotes":1,"from":1,"over":1,"Read":1,"Enjoy":1,"Poetry":1,"I,":2,"too,":2,"sing":1,"America":2,"I":3,"am":3,"darker":1,"brother":1,"They":1,"send":1,"me":2,"eat":2,"in":2,"kitchen":2,"When":2,"company":2,"comes":2,"But":1,"laugh":1,"And":3,"well":1,"grow":1,"strong":1,"Tomorrow":1,"Ill":1,"be":2,"at":1,"table":1,"Nobodyll":1,"dare":1,"Say":1,"Eat":1,"Then":1,"Besides":1,"Theyll":1,"how":1,"beautiful":1,"ashamed":1}
  3.  

Why jQuery.proxy rocks my world!

Posted on April 27, 2011, 2:09 pm, by Mayank Jain (makuchaku) and received 14 Comments ».

Its been very recent that I’m seriously looking into Javascript – and trust me, there is no dearth of stuff that can be achieved with purely client side code (HTML/CSS/JS). With the modern browsers churning out best of the breed in JS environments (both headed & headless), everything is going to come to a browser near you! Its just a matter of time!

Anyways, on more subtle issues, while writing a purely client side JS app using Backbone.js & jQuery Mobile I came across jQuery.proxy method – something that I’ve really started using almost everywhere.. The method is so useful that sometimes I wonder – why jQuery did not implement it in callback mechanism by default.

A quick overview about the problem…

  1. var FBNotificationWatcher = new JS.Class({
  2.  
  3.   initialize : function(timeout) {
  4.     this.url = "http://foo/return/json";
  5.     this.getNotifications(); // initiate the first fetch
  6.   },
  7.  
  8.   getNotifications : function() {
  9.     $.ajax({
  10.       // do your ajax call
  11.       url : this.url;
  12.       success : this.processNotifications
  13.     });
  14.   },
  15.  
  16.   processNotifications : function(rawNotificationData) {
  17.     // process notifications… AND
  18.     setTimeout(jQuery.proxy(this.getNotifications, this), this.timeout); // start all over again…
  19. });

When a new object of FBNotificationWatcher is created, all goes well. this.getNotifications gets the JSON & this.processNotifications processes it. However, when setTimeout’s occurs and this.getNotifications is called again, the context of “this” is set to the “window” object. And because of this, the code breaks in this.getNotifications method.

Now this is very frustrating – as you just lost the object you were operating within.

Solution? jQuery.proxy comes to rescue!

  1.  processNotifications : function(rawNotificationData) {
  2.     // process notifications…
  3.  
  4.     setTimeout(jQuery.proxy(this.getNotifications, this), this.timeout); // start all over again…
  5.   }

If we setup the setTimeout call like this, jQuery will guarantee that the context of “this” when this.getNotifications gets executed, will always be set to the original FBNotificationWatcher object which had setup the timer. The second argument to jQuery.proxy defines the context you want to get back when the setTimeout callback gets executed.

This concept can be used anywhere you want your callbacks to retain their context.

Read more about the method at jQuery API docs.

Kickass MCV for your Javascript Apps

Posted on April 9, 2011, 12:59 am, by Mayank Jain (makuchaku) and received No Comments ».

Very recently, I had an opportunity to work on a completely client side Javascript App – a Facebook stream reader. Since FB Graph API returns JSON, it can be directly consumed via JSONP. This app used JQuery Mobile for the UI screens (which totally rocks btw!).

I wanted to keep all the logic on the client end to as much extent as possible – with just a Sinatra component as a cross-domain proxy for post requests.

To my surprise, writing & consuming the Graph API via JSONP was the easiest – while managing the UI, writing the glue code between callbacks, data & UI was the messiest. I spent 3 days writing this spaghetti of callbacks coupled very tightly to the DOM ids/classes. At the end of 3rd day, it became almost impossible for me to extend the logic to include more functionality.

The code had become ugly – and I wasn’t too happy with this.

I started my search for a good MCV framework with a very simple objectives

  • Since I was already using JQuery Mobile, I did not wanted to program with another UI library
  • As lightweight as possible
  • Simpler learning curve
  • Should not get in the way of my programming style.
  • Well documented

I tried the most popular MCV frameworks – SproutCore & Javascript MCV – but didn’t really found them fitting my needs.

Then I came across – Backbone.Js – someone on Stackoverflow mentioned – “Thank god I found backbone.js”.

I took that review seriously & spent a night reading the API documentation. Even when I slept, my mind was trying to understand how I could use backbone.js in my architecture (wink wink).

I woke up 7 am the next morning & by 9 am, I had a very minimal architecture written.

By the time I wrapped up office today, I had completely re-written the App into a very nice looking MCV architecture, neatly placed into model/view/collection directories. Being able to re-render a view by just changing an attribute on the model object is a state of pure bliss.

I highly recommend checking out Backbone.js if you are even a wee bit serious about your Javascript App. Invest some time understanding how things are done with backbone.js … & I can bet that you will never want to directly tie your business logic to the DOM again!

Happy Hacking!

Tags:

How to commission developer machines in an agile way?

Posted on March 20, 2011, 1:35 pm, by Mayank Jain (makuchaku) and received 2 Comments ».

During the past year – working at AdoMado, I’ve seen myself & our developers wasting a lot of time in maintaining their own dev machines. Package install/upgrade/dependency failures, OS upgrade failures, hard drives failures, the machine not being available physically (in case of laptops), etc – are just a few cases.

Can there be a more agile way of working for developers? Given that the technology stack an organization runs on is often well defined, this should be fairly possible.

Very recently, I came across a project from one of the Slicehost founders Matt Tanase – BluePrint. The project lets you create versionable configurations from an existing installs. Just setup your machine the way you want it to be, run blueprint & wallah, you have a re-deployable configuration. Everything is created as a Git repository, hence any change you make to the configuration – is also versionable.

Next, if we can somehow automate the process of creating the machine images, we will be almost there. Luckily, there’s Vagrant. Its a tool to build & distribute virtualized environments. Blueprint also supports Vagrant.

Add to this, a web based editor (Bespin or Ace) – and you have a perfect recipe for dishing out pre-built images of your technology stack which are just perfect enough to be run on VirtualBox.

Imagine, its the first day of the developer. He can simply checkout such an image from a dashboard, run it on a VM & he’s ready to rock’n roll!

Since all of this is automated, this process can be improved & repeated as many times as one wants.

Sweet, aint it!

Ever since I started working on a Desktop as a full time machine, the only access to after-office-hours machine I have is my wife’s laptop – which being a WinBlows box, creeps me out. Such a setup, even if remotely possible – will give me the freedom I need to stretch my legs & utilize my free time :)

Just a single objective for 2011

Posted on March 13, 2011, 8:36 pm, by Mayank Jain (makuchaku) and received No Comments ».

Get adomado.com funded!

This year, there are no multitude of objectives, just this one.
The more focus I have, the better I can deliver.

From being employed to self-employed!

Posted on January 7, 2011, 9:45 pm, by Mayank Jain (makuchaku) and received No Comments ».

Its a huge huge step in my life – something I’ve always dreamt of ever since my college life – working for myself!

What I’ve always seen practically happening is once you burn the bridges, there’s only one way to go – and that is move forward!

That said, I still have butterflies in my stomach – but this time, I’m more determined than being scared of what’s in store for me.

Bring it on… head on!

« Older Entries