Project: Syncrow

Syncrow is a cross platform Node.js library for real time directory synchronization over tcp sockets. It can be used both as a node_module and a standalone command line tool. The idea came from my not so great experience with developing on remote environments over ssh. It started as a student project at the Jagiellonian University (Kraków), I got an A for a working prototype, but then I decided to turn it into something more serious, just for fun

Lesson learned: Tests make you move faster.

In the beginning I did not write a lot of tests, foolishly believing that Typescript will give me enough safety. This approach did not cause me any trouble during the prototype phase, but as the project has grown in size an complexity, often I broke things, and they remained broken for weeks, because only way to detect bugs was tiresome manual testing.

Then I stopped and wrote a test suite, not a great one, but it gave me at least some integrity guarantee. The test suite gave me 80% confidence that syncrow worked properly, without the need to test it manually. Back then I also wrote integration tests/benchmarks that simply checked how long does it take to transfer 10 big files (20mb) and 1000 small files (20kB) ?

Both the test suite and the benchmarks made me faster. I developed a habit that having written a couple lines of code, I ran the tests, knowing that, if some errors are caught they must be somewhere in those new lines. Before each commit I would run the benchmarks a few times to see, whether any performance issues has been introduced. During my next side project, I will develop the test suite simultaneously with the features.

Interesting bugs:

• Forgetting to remove a data listener from a stream, which caused disappearing of some files that should be transferred. Tricky to pin down, I had to track each socket connection and monitor its bytes read, to find where my leak was.
• Accidentally passing a result object as a first argument to callback in a long chain, which by Node.js convention should is reserved for errors.

JS Lessons

Code combining events and callbacks, comes with the risk that callback may be called multiple times. It is a good idea to wrap the callback with _.once to ensure that only first call is executed.

public static authorizeAsClient(socket:Socket, token:string, options:{timeout:number}, callback:ErrorCallback) {
    callback = _.once(callback);

    try {
        socket.write(token);
    } catch (e) {
        return callback(e);
    }


    socket.on('close', ()=> {
        if (socket.bytesRead === 0) return callback(new Error(`Socket has been destroyed - authorization failed`));

        return callback();
    });

    return setTimeout(
        ()=> {
            if (socket.destroyed && socket.bytesRead === 0) return callback(new Error(`Socket has been destroyed timed out`));

            return callback();
        },
        options.timeout
    )
}

Typescript lessons:

Typescript is a great tool, I really enjoyed using it. It eliminated a lot of stupid mistakes, and made refactorings much easier. Typescript still has some problems though: some errors found by Typescript compiler are not real errors, just mistakes in definitions of external libraries, including the Node.js core library

Constructor-property-shorthand in Typescript:

class MyClass{
  private age:Number;
  private name:String;
  constructor(name:String, age:Number){
    this.name = name;
    this.age = age;
  }
}

can be much simpler:

class MyClass{
  constructor(private name:String, private age:Number){}
}

I really feel that the free time I have devoted to this big side project has thought me a lot.