Google's heavy usage of Python, is it just a matter of taste or does it give them a competitive advantage?
-
23This needs to be community wiki– AutomatedTesterApr 1, 2010 at 12:38
-
72Alex Martelli incoming for the accepted answer real soon, ....– ChristopheDApr 1, 2010 at 12:41
-
5Actually I think this answer could be answered objectively (there must be statement somewhere by Google why they use it ;))– Felix KlingApr 1, 2010 at 12:41
-
15@nomemory, "heavy" is quite a correct adjective here (of course you'd need to be able to look at our codebase for confirmation -- how much Python vs Java vs C++ etc -- but, I am able to look at it, and contribute to it, mostly Python, most every day;-). And the question is not on all of the IT industry -- it's specifically on Google, a 20,000-employees company (including non-programmers), where some of us had better "have an overall view", since forming that view and acting upon it are core parts of our jobs;-). So, I'm voting to reopen.– Alex MartelliApr 1, 2010 at 14:28
-
7@nomemory, there's always keen Oracle recruiters at Python conferences so I'd be surprised to learn that Python is "non-existent" in their codebase (if so, those recruiters are lying, as well as wasting their own time and the Pythonistas';-). And anyway, how are Oracle and Google competing, really?-)– Alex MartelliApr 1, 2010 at 14:56
4 Answers
I can't really give a definitive answer, because by the time I interviewed at Google in 2004 Python was already prominent at Google.
Indeed, there's one apparently attractive explanation that I can definitely deny: it's not that Google uses Python because it employs so many prominent Pythonistas -- rather, most "prominent Pythonista" googlers joined Google, at least in part, because we knew about Python's prominence there (possible exceptions include Peter Norvig and Jeremy Hylton, but historically Google's choice of Python predated even them).
That's definitely why I first got interested (my publisher let me know about the large amount of copies of my book that Google was purchasing -- at first, I thought of it as a good opportunity to sell my freelance consulting service...;-), how I was later able to convince Guido to join us, and, I believe, part of the motivation for such Pythonistas as Greg Stein, Wesley Chun, Fredrik Lundh, Thomas Wouters, Collin Winters, Jeffrey Yasskin, ...
It all got started, I believe, because the very earliest Googlers (Sergey, Larry, Craig, ...) made a good engineering decision: "Python where we can, C++ where we must" -- they used (a subset of) C++ for the parts of the software stack where very low latency and/or tight control of memory were crucial, and Python, allowing more rapid delivery and maintenance of programs, for other parts. At the time, late '90s, the choice for the latter role was essentially between Python and Perl: other scripting languages were either unripe (I don't think Ruby was around yet, for example) or had other issues and limitations. Perl was more mature (especially in terms of its ecosystem of available add-ons via CPAN), but Python was deemed to be more readable and maintainable, and interfacing to C++ libraries (via SWIG) was easier.
Java came in later, covering an intermediate niche -- and more recently of course Go was developed (though I don't believe there's much production work in it yet, as it's still evolving and maturing). Some specialized languages such as sawzall are also in the mix for very specific tasks, and of course Javascript is very important for browser-side work.
Other languages, including the ones that Greg mentioned back in '06, are either "kind of accidental" or used for other specific tasks (e.g., Objective C for clients on iPhones or Macs) -- e.g., when Google hired its first system administrators, those employees inevitably came with very strong mastery of Perl and Bash, and often used either of those languages to develop some complex internal system; recoding those in Python (for easier deployment and maintainability) has often happened. Others (such as C#) may have been in the mix temporarily due to acquisitions, but, again, recoding in one of the "main Google languages" is always a pretty high priority (in C#'s case, recoding would typically be mostly in Java, as the two languages address similar areas in terms of levels of abstraction).
-
@Alex probably i didn't understand correctly but it seems google wanted their environment open for engineers (especially researchers) those who spent years in university and came out only C or C++ on hands (as everyone beginning with these languages and most of them ended up with it too). Here, python is strong contender provides an interface to huge lib of C/C++. Apr 12, 2010 at 11:57
-
8Time travel FTW: web.archive.org/web/20091222094118/http://labs.google.com/… Apr 17, 2012 at 3:47
-
3One tidbit I collected from reading Levy's excellent book "In the 'Plex" is how Python got into Google (before it was named Google -- it then was "backrub"). Larry Page had written a web-scraping "spider" in then-brand-new Java 1.0 beta, and it was so broken that his dorm roommate helped him by rewriting it in a more staid, solid and productive language... Python. Funny to think that, without that "spider", Google might never have come into existence... Jan 1, 2015 at 16:59
I have not read the whole article and I don't know how representative it is but maybe this answers your question: Python at Google.
Ok I read most of it, unfortunately it gives not that much insight but I think it is still an interesting article. Maybe most important:
At Google, python is one of the 3 "official languages" alongside with C++ and Java. Official here means that Googlers are allowed to deploy these languages to production services. (Internally Google people use many technologies including PHP, C#, Ruby and Perl). Python is well suited to the engineering process at Google. The typical project at Google has a small team (3 people) and a short duration (3 months).
Not to forget that Guido van Rossum the creator of Python worked for Google from 2005-2012 ;)
I also found this quote but I cannot verify it:
"Python has been an important part of Google since the beginning, and remains so as the system grows and evolved. Today dozens of Google engineers use Python, and we're looking for more people with skills in this language"
-- Peter Norvig, Director of Search Quality at Google
-
-
4@nomemory: Maybe both? It fits the engineering process (taste) and due to that might give them an advantage if they can finish their products earlier. Apr 1, 2010 at 13:16
-
2Dozens of engineers use Python? For a company as big as Google, that's hardly a compelling case. Dec 15, 2011 at 22:48
I was reading about Unladen-Swallow, a project that aims to improve the performance of CPython. While browsing through their discussion board I stumbled across this discussion, where Collin Winter (Google employee and Python core developer) answers on the claim that Google employees are being discouraged from using Python for new projects:
Well, simple common sense is going to limit Python's applicability when operating at Google's scale: it's not as fast as Java or C++, threading sucks, memory usage is higher, etc. One of the design constraints we face when designing any new system is, "what happens when the load goes up by 10x or 100x? What happens if the whole planet thinks your new service is awesome?" Any technology that makes satisfying that constraint harder -- and I think Python falls into this category -- should be discouraged if it doesn't have a very strong case made in its favor on other merits. You have to balance Python's strengths with its weaknesses: your engineers may be more productive using Python, but if they have to work around more platform-level performance/scaling limitations as volume increases, do you come out ahead? etc.
If you subscribe to Paul Graham's idea that succinctness == power (i.e. If I can write code in language X in 10 lines to accomplish what you took 100 lines of language Y, my language is more powerful)
These days hardware is cheap, meatware is expensive. I know that I can write a lot more, faster, better AND clearer in Python than I can in other languages, simply because the batteries tend to be included. Sure, it may be slower but are you really going to notice the difference between .05 seconds and .0005 seconds?
In a more real-world example, if Google developers write a program in Python where they never have to worry about garbage collection, and Company X writes a program in C++ and they spend 10% of their time tracking down bugs related to memory allocation, guess who's done faster - even if the C++ shop manages to write the rest of their code just as fast.
So yes, I'd say it is a competitive advantage to code in Python.
-
2Cursed site redesign! Well he's got an alternate link, so there ya go. Jul 22, 2010 at 20:18
-
12'are you really going to notice the difference between .5 seconds and .005 seconds?' You don't work on Google core search, YouTube or Gmail, do you?– jwgApr 19, 2013 at 13:24
-
6
-
6The great thing about Python is, when you have a significant performance difference (e.g. ~495ms), you can plug in some C/C++ code. Python allows you to develop quickly and optimize efficiently.– TrevorJan 23, 2014 at 19:55
-
3The difference between .5 seconds and .005 seconds is that one is noticeable and the other is not. Feb 15, 2014 at 14:31