After several weeks of limited development due to finals in school, I have been able to sit around for several days and completely rewrite my API for the third time in Python. My first version was rewritten in Object Oriented Perl, however I found that I could not easily do everything I wanted to do with Perl, and I discovered the Web.py project for Python and fell in love. I also found CoffeeScript and have finally began to work a lot with it.
Web.py is a pretty powerful, small web frame work for Python. It makes development of a site a pinch, and allows for tons of features, such as very good (in my opinion) MySQL support, URL routing and an easy way to work with GET and POST requests. For example, Web.py proudly displays this short snippet on it’s homepage:
import web urls = ( '/(.*)', 'hello' ) app = web.application(urls, globals()) class hello: def GET(self, name): if not name: name = 'World' return 'Hello, ' + name + '!' if __name__ == "__main__": app.run()
I mean, come on, thats too easy. Tie that in with the Python Standard Library’s JSON module and you have the perfect building platform for a simple, concise and easy to manage and follow API.
API: Project Blue Ring
So with the internet acting like my own Prometheus, I set to work rewriting the API in the language I love.
It wasn’t soon before I had the basic structure finished and the Python looking comparable to the Perl version, only one thing was different: The URL’s. For the Perl versions, I had to always stick with an ugly URL that looked something like this
Who would ever want to type that in? Instead of working with mod_rewrite, which would have required me to change apache’s config every time I changed the URL scheme, I choose to stick with it and forget. But this wasn’t acceptable when I saw the power of a few simple lines in Web.py:
urls = ( '/(.*)', 'hello' ) app = web.application(urls, globals()) class hello: def GET(self, name): return something def POST(self): return something_again
This meant I could do a URL more like:
/product/dog987/info/ and >/product/add/
One which takes GET requests and the other which takes a POST request (due to the data being sent). Isn’t that easy to read and use? ~nods~
So, using this new knowledge of how to make fire, I set out to build more (or burn more down, to stick with the metaphor…). Working hard into the nights I burnt a lot of midnight oil working on getting Python to predict when a given product will run out based off of previous withdraws from the stock. I didn’t want to use a module like I did in Perl (Statistics::LineFit) and instead really wanted to be able to use R, which I had written a small script in that predicted this value, inside of Python. So I found rpy2. Rpy2, it turns out, is really nice and quite powerful, I even rewrote my pure R script in python with rpy2. But like all great things… it didn’t work. Instead, when placed in a Web.py app, it hung. Even with every log and debug setting turned on, I got no errors, the web page would simply hang in reload for infinity. That was a problem because that was the heart and gold of this API currently, being able to predict when a product’s stock will run out.
The whole rpy2 fiasco ended up costing me several weeks, during which, little work on other parts of the API were completed. Because I could not get it to work for the life of me, I choose to abandon rpy2 and R, and implement my own version of the math in as pure Python as I could get. As a result, after a few days of more midnight oil burning, I built my own Vector type for Python.
Great! now what does that mean and what does it do?!
The way the prediction is being made, is by forming an equation, however instead of just normal numbers, the variables are in fact matrices or in this case, Vectors, sort of like an array. As a result of python lists not being able to fully complete what I needed at the time, the Vector type worked perfectly, and still does. Add another week onto this list and you end up with the amount of time it took me to build the first step of the stats function and remove most if not all errors I can generate on my own.
With this all done, it was time to add pictures to the products, so I choose to rewrite the add and update functions as POST only requests which are capable of uploading a picture, then using the Python Image Library (PIL) I wrote a small thumbnail generating function and wrapped it all up inside of a nice class to add more features to later on. To test the POST picture uploading, I choose to use the poster library for Python which as it turns out, handles everything perfectly also. I did look into httplib2, and requests, however I found that none of them made streaming the picture file easy, where as poster did.
At this point, it was back to smooth sailing. I had Python giving JSON replies to everything from total inventory, adding, updating, deleting to viewing usage logs, and predicting when a product will run out; the URL scheme was perfect, and everything was working fine. Good enough for a first version I figured, now time for a basic web GUI to wrap up the first RC for version 1.
This came up with a few interesting things, as I had never down uploading a file via a POST request with a web page, only through things like poster for Python. I turned to the AJAX method of using a hidden iFrame to submit a form, and it turns out, it works perfectly! So with a basic web admin panel almost all the way working (only missing the update, stats and log functions for products) Project Blue Ring is almost ready for v1RC1!
You can find more info on the project here: https://github.com/JoshAshby/House-Inventory-API