post

[Tech] Why is “Database Layer as a REST API” not common?

We have “database APIs” such as abstraction layers over multiple SQL databases and ORMs. But why not take it to the next step and make it a REST API like any other network call that we can make?

Database as a REST API

Database as a REST API

Advantages would be:

Did we just sort-of reinvent Datomic?

Of course, this is not a new idea at all, take restSQL as an example – my question is why is this not talked about more often?

Do most frameworks support this? If not, why not? If so, why don’t most frameworks don’t talk about such a use case in their documentation? If I use Django, I’ll start writing the models and use South to create migrations, and that’s that. If I have to reuse those model, from say, Java, then you’re on your own. The point is that, by default, Django (or Rails) doesn’t encourage you to do such a thing. If you go for a lighter framework such as Flask, then this becomes easier because the ORM is anyway not part of the framework.

Is this concept felt needed only in a polyglot case (multiple database systems, multiple programming languages)?

P.S. Also read Stevey’s Google Platforms rant.

Update on [2013-04-28 Sun]: Also see the very useful tech talk Designing a Beautiful REST+JSON API.

Advertisements
post

Wrote an EDN format reader and writer in Python

I was reading about the EDN format over the weekend. EDN (pronounced like in “eden garden”) is a data format in the same league as JSON but is supposed to have some nifty features such as sets, keywords, date-time type, custom types, and also being a proper subset of Clojure.

Having a date-time type as well as custom types seems useful to me, so I was taking a look at the current Python implementations of the EDN format and I didn’t find them satisfactory, for example, one of the listed ones had all custom parsing code which was difficult to read, one was not even a real implementation, just boilerplate code, etc.

So I thought why not create a better implementation and I did – it is up on GitHub at https://github.com/swaroopch/edn_format.

It has been a long time since I did lex and yacc, so it was a fun weekend project :)

post

A Byte of Python now available as EPUB

Some people, when confronted with a problem, think “I know, I’ll use a wiki” Now they have no idea how many problems are there.

Andrew Clay Shafer

The above quote summarizes my experience in the past years in having converted A Byte of Python book into a wiki. I was hoping to make it much easier for readers to make corrections and contributions to the book. The only thing it enabled is tons of spam.

The second issue was that lots of readers kept emailing me to ask about ebook and kindle versions of the book which I could not do because it was stuck in a wiki format.

The solution was obvious – Pandoc, but it just seemed too daunting a task to do and hence I delayed it for years. It has been done now, thanks to my wife who did the conversion from the earlier Mediawiki syntax.

I also wrote a small Fabric file to update the websites in a single command invocation – so now I can edit the book and by running one command, it will update the book chapters pages on the WordPress site as well as generate and update the PDF and EPUB files stored in AWS S3.

I have also made a few quick changes to the text:

  • Overhauled installation and first steps chapters for Python 3 and explaining how to find and open a terminal application in detail
  • Recommending newbies to start with ActiveState Komodo Edit editor and instructions on how to use it to create and run a source file
  • Removed unnecessary sections such as nonlocal, metaclasses, exec and eval, etc.
  • Moved escape sequences in strings to the ‘More’ chapter in the end, it was an unrequired hurdle

I haven’t done an exhaustive review of the text yet, because frankly, that is draining, but I hope this is the start of me responding to readers’ suggestions on how to improve the book.

Enjoy the new EPUB and PDF formats. The sources are at http://github.com/swaroopch/byte_of_python. Kindle edition, etc. coming soon.

post

Everyday reminder about Impact

Everyday I get emails from readers of my books that makes me wonder: Am I continuing to make an impact everyday? Can I have the same impact again through my work? Maybe, let’s see.

But still, it’s difficult to top this:

I’m a 16-year-old boy from Guangxi,China.

Thanks for the book named A Byte of Python of yours.I learnt lots of things from it.

And this:

Dear Sir,

I have never been able to program before but with your byte of python I actually managed to not only write the small programs but even understand. For me this is good because I am 57 years old and I always gave up any programming I attempted to learn before no matter how desirable a goal it seemed. The real problem I run into is what the words mean. I consider myself fairly literate but when I run into a special word I do not know the meaning of, it frustrates me, especially when I cannot find a definition, and there goes the understanding. Also, there are gradients of knowledge that are skipped. One minute you’re riding along nicely and the next minute you’re behind the 8 ball. Along the way, one quits. You, on the other hand, seem to have undercut this better than anyone I have encountered before. Your explanations are clear and concise and you define words well so I understood more of python than I have of any other language including Basic which I tried and gave up on years ago and the understanding flows very well. And you’re giving hope to an older person who really wants to learn to program!

Thanks so much

These emails are enormously humbling and reminds me to continue to focus on the impact of what I do.

post

Programming Language Adoption

55 people have asked me to write about:

Indian companies just don’t take Python seriously, not as much as Perl. Why is that?

28 people have asked me to write about:

Most companies opting for Ruby as their programming language of choice instead of Python

51 people have asked me to write about:

The future of the Perl programming language?

Phew, that is quite a lot of people asking about programming language adoption!

The TLDR version of my thoughts on this topic is: Companies are choosing the right tool and the right community for the right job. Which programming language is used in your company depends on the kind of work they are doing.

Ruby and Rails ecosystems are built by and built for web programmers. That is why you will see web programmers switching from PHP and Python to Rails. For web programming, I do believe that Rails has an edge, not just because of the framework itself, but because of the community rallying around it. The amount of amazingly useful stuff that they churn out outpaces any other community w.r.t. web-dev, and they even have quick effective screencasts to make it digestable, hence their edge!

Data analysis is the hot thing these days and as per O’Reilly, Python is the choice of language for data analysis even though Ruby gets more buzz in the tech news. It is the same reason why scientists are switching away from Ruby to Python.

Similarly, I believe that Perl is still used a lot as a “glue language” and “text munging language” which has always been its forte and continues to be its forte, although Python and Ruby have been slowly entering that territory. Even though there are marketing reasons why Perl is no longer as popular, if you do use Modern Perl, you can be as effective as with the other two languages [1]. For example, if you notice the recent announcement of Amazon “Simple Email Service”, you will notice that their scripts SDK is written in Perl! Also, if you read the latest Perl news it is not lagging behind the other languages, it just doesn’t get the “buzz” factor these days. Programming languages are like fashion, they keep coming back in cycles. Remember the days when JavaScript was considered a pain and today, it’s the new hotness? (same goes for Haskell) We owe a lot to Perl and perhaps Perl 6 will teach us the future of things yet again, who knows!

So I have a question back for the 134 people who asked the above 3 questions – Is the programming language chosen by your company match the kind of community we have described here?

What would you say to these 134 people?

[1] Perl was the original magical language which I still have a soft corner for, since I have written a lot of Perl code at Yahoo! and thoroughly enjoyed it.

post

Coding Problems for Homework

“Coding Homework” is a small website that I have built to list small problems that one can work on, to learn how to use a new programming language. For example, small problems requiring to read from a file, or to use regular expressions, how to find duplicate files in a folder, and so on.

Note that the problems listed on the site is not for testing your algorithm skills, there are many sites for that already.

This list was inspired by repeated requests and suggestions from readers of A Byte of Python for homework problems at the end of each chapter to exercise the skills they have just learned. So I thought why not make it applicable to any language and multiple programming skill levels. And it’s a good topic that can be collaboratively worked on with the programming community, à la Stack Overflow.

All the content will be licensed under Creative Commons Attribution-Share Alike 2.5 India License so that anybody can reuse this content, especially in classroom situations.

Screenshot of "Coding Problems for Homework" website

I also had my own specific goals when implementing this side project:

  1. Solve the lack of “homework problems” for people to exercise their programming skills, especially in the context of learning a new programming language.
    • I am not trying to replace existing lists but rather focus on making the reader active (providing exercise problems) than letting him/her be passive (reading code listings).
  2. Learn how to do website layouts, specifically how to use YUI Grids CSS.
  3. Learn how to pick colors for website design; ColorCombos turned out to be useful.
  4. Learn to use Google App Engine.

It has been a fun side-project, spending a few hours here and there. It is very far from polished, but the basic functionality works. There is still more to do — adding a search functionality, conforming to standard UI design patterns, caching for the rendered HTML (from Markdown), optimizing the housekeeping code, and so on.

This site itself is a good example on the kind of problems that beginners can work on, but they would not know what kind of problems they can solve and what level of expertise (beginner / intermediate / advanced) would be needed. That is where this list of problems can help.

I request you to spend 5 minutes of your creativity to add a few problems so that beginners and intermediate level folks will have interesting problems to test their learning of a new programming language. Thanks!

It might be helpful to you as well when you’re going to play around with functional languages (Haskell, Erlang, etc.), funky new languages (Ioke), or new languages by big companies (Go).

Link: http://codinghomework.appspot.com

post

An interactive version of A Byte of Python

Roorky is a new startup that has created a new file format and software for interactive books. As part of the default installation of the software, they are bundling A Byte of Python free with the software.

What is interesting is that this may help complete beginners who stumble in getting started with IDLE, etc. The most common email that I get asked is when people run python abc.py on the IDLE interpreter prompt and wonder why it is not working – both the concepts of two command lines as well as the concept of running a program is not grokked well by beginners who are getting started, especially people who are self-taught. From that perspective, this is an interesting approach to the problem.

I am still not convinced about this approach because people cannot bypass the learning of how to edit, compile and run the code in the native environment, because that will be needed when writing new programs. It will be interesting how the two opposing needs will be balanced.

But I hope beginners will try it out and see if it helps them get started.

A full size video of a walkthrough of the software is available at the Roorky website.

Note: Be warned that it is a beta software.

post

Interview with CocoaCast

I was recently interviewed by CocoaCast (mp3) to talk about Python for their Mac developer community, as part of their “Unbound Developers” podcast series.

I was apprehensive about doing this because I’ve never been live-interviewed before, but in the spirit of doing new things, I went ahead and it turned out to be fun.

We talked about Python, my background and my startup, Macs, Python vs. Ruby and Django vs. Rails, Apple and Google, cloud computing, databases, Scala/JVM, performance, collection classes, and so on.

Surprisingly, Vlad and myself had talked for nearly 25 min. It’s a fun interview, although, there’s probably nothing new to know from this podcast for those who have already read the book.

post

Why Stack Overflow is useful

In one of my previous thoughts, I had mentioned about a website called “Stack Overflow” in passing.

I’m surprised that many people do not know or use this resource and community. Just a brief search over there would answer so many questions that programmers have.

For example, there is a suggestion on my skribit page:

“how to give back to the open source community”?

And this question has 20 votes!

I wonder why this question to me, then I remembered this suggestion popped up soon after I wrote “Why use Creative Commons license?” where I had written “The book was intended to be a contribution back to the open source community. We constantly keep taking and taking – whether it is using Linux, Vim, Firefox, or countless other software, so it felt great to be useful to the community in return.”

I guess I had it coming.

First of all, I would say that the best place to actually learn such a topic would be another book (I bet you saw that one coming!) called Producing Open Source Software by Karl Fogel (which is itself an open source book) to understand how an open source project works right from the technical infrastructure to the social and political infrastructure, how to communicate, and so on. And finally, the chapter on Volunteers explains the different kinds of volunteers that are helpful to an open source project which indirectly means that those who are interested can participate in the projects in one of those roles => You’re giving back to the open source community!

But perhaps there are better suggestions in this discussion on Stack Overflow when somebody asked, duh, How to get involved in an open source project?

Someone also posted another skribit suggestion asking:

hi, can u give me a link on examples with python or projects in python book i am a beginner

Guess what? I already answered that in a discussion at Stack Overflow.

The answer is that there are two projects – the “Programming Language Examples Alike Cookbook” project and the Rosetta Code project which lists vast numbers of example programs in multiple programming languages.

Again, the person could have found this answer already by a simple search on the Stack Overflow website.

For some of the programming queries I had, I didn’t know whom to ask. There used to be an internal algorithms-discuss mailing list when I was at Yahoo!, but whom do I turn to now? The answer again was “Stack Overflow” (which at that time was yet to be launched, so I was waiting in anticipation):

How to convert floats to human-readable fractions?

Let’s say we have 0.33, we need to output “1/3”.
If we have “0.4”, we need to output “2/5”.

The idea is to make it human-readable to make the user understand “x parts out of y” as a better way of understanding data.

I know that percentages is a good substitute but I was wondering if there was a simple way to do this?

And someone nicknamed “Epsilon” pointed to me to a brilliantly simple algorithm by David Eppstein which exactly answers this question.

# Usage: ./frap <fraction> <maximum denominator>

$ ./frap 0.33 10
1/3, error = -3.333333e-03
3/10, error = 3.000000e-02

$ ./frap 0.2342 100
11/47, error = 1.574468e-04
15/64, error = -1.750000e-04

Isn’t that amazing? Both the algorithm and the community at Stack Overflow.

If you want to know why Stack Overflow works so well, there is a discussion on that, at Stack Overflow you might want to read :-)

post

A Byte of Python in hard copy

If you’re the kind of person who prefers to read a physical book vs. online books, then you’ll be happy to know that the A Byte of Python book is now available as a printed hard copy.

The best part is that the hard-working translators can also publish their translations and sell the printed copies, benefiting both the readers and the translators.

I had received many requests from readers for hard copies of the book and I’m glad to finally get this working. Interestingly, I was previously trying to get the book printed via CreateSpace because the book would automatically get listed on Amazon.com (since CreateSpace is owned by Amazon). However, their process was not streamlined and confusing. Worse, I couldn’t get the PDF in their required size formats because of a bug with mwlib.rl.

I got tired and decided to try Lulu and I was very surprised. They are miles ahead in terms of usability of their service as well as wide range of options and sensible defaults. For example, it was a pain waiting for manual approval of the book by the CreateSpace staff and it is an unnecessary delay every time I upload a new version. On the other hand, Lulu made it very easy to design a rudimentary cover using their process. Overall, I was able to make the printed copy available for purchase in a single evening.

Of course, all this is possible because of the ability to generate PDFs from a wiki, thanks to the nice people at PediaPress.

Update: For Indian readers, the book is now available via pothi.com.