Dependency management with CoffeeScript and Sprockets

Sprockets makes it really damn easy in a few lines of ruby to concatenate and serve your coffee script files. It supports multiple template engines ERB, JavaScript, CSS, SCSS, etc you can even add a dash of ERB <%= %> to your coffeescript source files by naming your file source.coffee.erb and sprockets will automagically compile it down with ERB then CoffeeScript and concatenate  it with other dependencies!

For the project I was working on I couldn't depend on Node.js for dependency management since I was creating a front-end browser game. At first I decided to implement my own hacked up solution in Ruby to substitute and concatenate coffee script source files... it wasn't pretty :( After moving to sprockets I was able to refactor a few hundred lines into three :)

For example, suppose you have a file that contains sprockets `require` syntax called Main.coffee

#= require "src/one"
#= require "src/two"
#= require "src/three"
MyApp ?= {}

Then all you need to do is create a Sprockets::Environment instance and add the source path

require 'sprockets'
env = Sprockets::Environment.new
env.paths << "src/coffee"
open("example.js", "w").write( env["Main.coffee"] )

Tada! It just works :)

Stubbing out enumerable :each in Rspec

Ran into a tricky situation the other day trying mock out an enumerable object. I was trying to test a list of a files inside a Zip. However, I couldn't figure out how to yield a list of mocked ZipFile objects without monkey patching something inside Rubyzip which didn't seem very attractive at the time.... Instead I decided to simply stub! the :each method and create a helper

  def mock_out_enumerable_each(*files)  
    block = lambda {|block| files.each{|n| block.call(n)}}
    @zip.stub!(:each).and_return(&block)
  end

This works rather nicely, 

one = mock('one', :name => "assets/file1.jpg")
two = mock('two', :name => "assets/file2.png")
three = mock('three', :name => "assets/file3.gif")
four = mock('four', :name => "assets/file4.gif")
stub_out_enumerable_each one,two,three,four  

ReviewBoard with Nginx

This weekend I decided to checkout ReviewBoard a slick Web 2.0 open source Django app for doing code reviews. Unfortunately, it took me quite a bit of time to set it up from scratch with Nginx. Thus, I decided to do a quick writeup for Nginx users.

ReviewBoard's setup docs is a good starting point for getting the dependencies you need to setup a ReviewBoard website.  Once you have ReviewBoard installed you can generate a website using `rb-site install`. I setup my website under /var/www/review.mysite.com but you can set it up anywhere if you prefer another location. 

The next step is to launch the FastCGI daemon script bundled with ReviewBoard

rb-site manage /var/www/review runfcgi method=threaded port=3033 host=127.0.0.1 protocol=fcgi 

Once your FastCGI instance is up and running you just need to simply point Nginx at the FastCGI process 

server {
    listen 80;    
    server_name review.mysite.com;
    root /var/www/review.mysite.com/htdocs/;    
    
    location /media  {     
      root /htdocs/media/;    
    }
   
    location /errordoc {      
      root /htdocs/errordocs/;    
    }

    location / {      
      # host and port to fastcgi server      
      fastcgi_pass 127.0.0.1:3033;      
      fastcgi_param PATH_INFO $fastcgi_script_name;      
      fastcgi_param REQUEST_METHOD $request_method;      
      fastcgi_param QUERY_STRING $query_string;      
      fastcgi_param CONTENT_TYPE $content_type;      
      fastcgi_param CONTENT_LENGTH $content_length;      
      fastcgi_pass_header Authorization;      
      fastcgi_intercept_errors off;    
    }  
}

Then restart your nginx server and reviewboard should be up and running. Did I miss anything?

Producer-consumer problem in C

For my Operating Systems class CSCI360 we had to write the classic Producer-consumer problem in C using semaphores. This is a classic problem/solution to preventing race conditions over a shared resource.  For those of you who don't know what a semaphore is it's simply a variable that acts as a counter ( with two operations up/down ) which puts a process to sleep if it tries to down/decrement a 0 valued semaphore. The process will receive a wakeup signal once the 0 valued semaphore is up/incremented by another process.

In the producer-consumer example below we have a critical region (the shared buffer) that needs to be controlled using semaphores to prevent race conditions. We create two semaphores one called EMPTY which represents the number of empty slots available for the producer and another semaphore called FULL to represent the number of slots occupied with items.

If there are no items in our buffer a consumer will goto sleep since it will try to down the FULL counter which will be 0 because there are no items occupying any slots for it to consume. Once the producer creates an item it will down EMPTY and up FULL; vice-versa. In the case of the consumer when it consumes items it will up EMPTY and down FULL. This might seem a little confusing at first so I created a diagram to illustrate the process (note: producer/consumer are NOT running in parallel in this diagram).

It is important to note that EMPTY and FULL alone do not prevent race conditions.  This doesn't cover the case when the buffer is neither full nor empty. This means we need a third semaphore to lock the shared buffer to prevent consumers and producers from accessing the shared buffer at the same time. We call a semaphore with a single counter (initialized to 1) a binary semaphore or more commonly referred to as a mutex or lock. Using a mutex initialized to 1 when a consumer wishes to consume an item it simply needs to down the mutex before consuming the item which will prevent other producers/consumers from manipulating the shared buffer.

There are two main semaphore implementations in UNIX-based systems: System V (available in older-systems) and POSIX. For my example program I used System V implementation.

Producer

#include "shared.h"

void insert_item(int item, int semid, int *shared_buffer) {
  int index = get_buffer_size(shared_buffer);
  shared_buffer[index] = item; 
}

int produce_item() {
  return 0xFF; // nothing dynamic just write a static integer a slot
}

int main(int argc, const char *argv[])
{
  int *shared_buffer = create_shared_mem_buffer();
  int semid = create_semaphore_set();

  clear_buffer(shared_buffer); // prepare buffer for jobs

  int item = 0;

  while(1) {
    item = produce_item();
    semop(semid, &downEmpty, 1);
    semop(semid, &downMutex, 1);
    insert_item(item, semid, shared_buffer);
    debug_buffer(shared_buffer);
    semop(semid, &upMutex, 1);
    semop(semid, &upFull, 1);
  }
 
  return EXIT_SUCCESS;
}

   

Consumer

#include "shared.h"

void consume_item(int item) {
  // do something with item
}

int remove_item(int semid, int *shared_buffer) {
  int index = get_buffer_size(shared_buffer) - 1;
  int item = shared_buffer[index];
  shared_buffer[index] = 0x00;
  return item;
}

int main(int argc, const char *argv[])
{
  int *shared_buffer = create_shared_mem_buffer();
  int semid = create_semaphore_set();

  int item = 0;

  while(1) {
    semop(semid, &downFull, 1);
    semop(semid, &downMutex, 1);
    item = remove_item(semid, shared_buffer);
    debug_buffer(shared_buffer);
    semop(semid, &upMutex, 1);
    semop(semid, &upEmpty, 1);
    consume_item(item);
  } 

  return EXIT_SUCCESS;
}

 

shared.h

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
#include <sys/shm.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

#define BUFFER_SIZE 5
#define EMPTY_ID 0
#define FULL_ID 1
#define MUTEX_ID 2
#define NSEM_SIZE 3

#define SHM_KEY 9
#define SEM_KEY "."

static struct sembuf downEmpty = { EMPTY_ID, -1, 0 };
static struct sembuf upEmpty = { EMPTY_ID, 1, 0 };
static struct sembuf upFull = { FULL_ID, 1, 0 };
static struct sembuf downFull = { FULL_ID, -1, 0 };
static struct sembuf downMutex = { MUTEX_ID, -1, 0 };
static struct sembuf upMutex = { MUTEX_ID, 1, 0 };

int *create_shared_mem_buffer();
int create_semaphore_set();
int get_buffer_size(int *sbuff);
void clear_buffer(int *sbuf);

 

shared.c

#include "shared.h"

/**
 * returns current size of shared buffer
 */
int get_buffer_size(int *sbuff) {
  int i = 0;
  int counter = 0;
  for (i = 0; i < BUFFER_SIZE; ++i) {
    if (sbuff[i] == 0xFF) {
      counter++;
    } 
  }
  return counter;
}

void debug_buffer(int *sbuff) {
  int i = 0;
  for (i = 0; i < BUFFER_SIZE; ++i) {
    if (sbuff[i] == 0xFF) printf("1");
  }
  printf("\n");
}

/**
 * returns a pointer to a shared memory buffer that the
 * producer can write to.
 */
int *create_shared_mem_buffer() {
  int *shmaddr = 0; /* buffer address */
  key_t key = SHM_KEY; /* use key to access a shared memory segment */
  
  int shmid = shmget(key, BUFFER_SIZE, IPC_CREAT | SHM_R | SHM_W); /* give create, read and write access */
  if (errno > 0) {
    perror("failed to create shared memory segment");
    exit (EXIT_FAILURE);
  }

  shmaddr = (int*)shmat(shmid, NULL, 0);
  if (errno > 0) {
    perror ("failed to attach to shared memory segment");
    exit (EXIT_FAILURE);
  }

  // clean out garbage memory in shared memory
  return shmaddr;
}

/**
 * only used in the producer to clean out garbage memory when
 * constructing initial buffer.
 */
void clear_buffer(int *sbuff) {
  int i = 0;
  for (i = 0; i < BUFFER_SIZE; ++i) sbuff[i] = 0x00;
}

/**
 * create FULL and EMPTY semaphores
 */
int create_semaphore_set() {
  key_t key = ftok(SEM_KEY, 'E');
  
  int semid = semget(key, NSEM_SIZE, 0600 | IPC_CREAT);
  if (errno > 0) {
    perror("failed to create semaphore array");
    exit (EXIT_FAILURE);
  } 

  semctl(semid, FULL_ID, SETVAL, 0);
  if (errno > 0) {
    perror("failed to set FULL semaphore");
    exit (EXIT_FAILURE);
  }

  semctl(semid, EMPTY_ID, SETVAL, BUFFER_SIZE);
  if (errno > 0) {
    perror("failed to set EMPTY sempahore");
    exit (EXIT_FAILURE);
  }

  semctl(semid, MUTEX_ID, SETVAL, 1);
  if (errno > 0) {
    perror("failed to create mutex");
  }

  return semid;
}

 


View older posts »

Github