A couple weeks ago, we had another hackathon at work. This time, I wanted to create something useful that could potentially be adapted by one of our teams. My initial thought was to convert some of our old C++ code to Go and remove a Windows dependency, but after talking with one of my coworkers, we decided to instead create a DNN service for executing a Caffe2 model in Go. We had a third volunteer join us and we set out to achieve our goal. What we ended up with was a Python service that uses PyTorch to execute a neural network.

So what happened? We started with a Go service, complete with a gRPC proto file. Clients were generated to talk to the service in Go, Python, and C++. Then when it came time to actually consume Caffe2, which is written in C++, we realized that most documentation out there is in Python. I had not actually used Python myself before this time, but one of my coworkers is a pro, so we decided to generate gRPC server endpoints in Python and whip up a Python server. It came together surprisingly fast. We were trying to convert one of our internal models to Caffe2, but weren’t quite there yet, so we used a sample model that was already trained and ready to go with regular PyTorch. Given even half a day more, we would have had an actual production model running on our Python server with Caffe2!

One of my contributions to this project, besides cleaning up the server code and making it more robust, was a Dockerfile I created for running the service. We were of course able to run the service locally, but we wanted to be able to run it in a Docker container, since we use AWS ECS for hosting our services. Not only did I make the Dockerfile for the server, but I created it with a parameter so that the container can be built with any model. Instead of creating specific DNN services for each model we need to run, we can use the same generic DNN service and build the container with a different model. This will allow our labs team to deploy newly trained models quickly and confidently.

This hackathon did not turn out how I initially envisioned it, but it did turn out to be a lot of fun and our team was one of the few teams with an actual working prototype after the two days were finished. It’s very probable that some form of our work will end up in production at some point in the future. We garnered a lot of interest when we showcased our work at the conclusion of the hackathon. I personally really enjoyed jumping into Python, which is really easy to learn. There were a few frustrations, but all in all, 10/10, would do it again!