Technologies @ Pearltrees

[This post has previously been published on Nicolas Cynober's blog. Nicolas is a R&D Engineer and co-founder of Pearltrees]

Many of you asked us about the technologies behind all these pearls. The web, as much as the internet or even computer science is a succession of technologies stacked one on another, so we’ll try to list here some of the essential technologies we use as well as to represent our production architecture in a simple way.

 

User Interface

Flex

Flex is an ActionScript (Flash) Framework that enables us to develop the user interface of the web version. It makes it possible to quickly develop a rich interface and to deploy it on a large number of terminals. Today, our main challenge is to manage the size of the generated application which is a key element in app loading time.

HTML

Essentially used to feed the search engines and to give an alternate version for terminals that don’t support Flash. All the content in Pearltrees is available in HTML format. We also worked a lot on HTML when developing the Pearltrees embed. To help us resolve integration issues in an uncontrolled environment we are using HTML5 properties.

Javascript

We are using javascript in many cases: animations in Pearltrees’ browser, embeds and extensions. We are using JQuery when manipulating the DOM.

RDF

RDF is a semantic web related format that we use for exporting data from Pearltrees. Instead of creating another XML scheme, RDF is a format better adapted for sharing linked data which is crucial since pearls are both linked together and linked to URLs.

XUL

XUL is the presentation format used by Mozilla. We use it in Pearltrees’ extension for Firefox.

C++

The IE extension has been entirely developed in C++.

Objective-C

Soon, you will know why we have used Objective-C ;) We use the cocos2d game framework to handle various animations and effects.

 

Server

Java

We are using Java 6 and Tomcat in order to make the Pealrtrees application and our ranking algorithm “Tree Rank” work. Thanks to its “secret recipe”, Tree Rank enables us to calculate the relations between the pearltrees and to make it easy to discover related contents. Tree Rank is running on an HazelCast cluster and is hosted in the Cloud. We also are using Lucene as a basis for our search engine and we are currently experimenting with Cassandra.
Initially we tested PHP, but Java is now used in most of our server software.

PHP

Our backend is still using PHP. First on Zend Framework and Apache, most of our server code is now on Java. We decided to use java instead of the Zend framework because we had performance issues with the latter. However, we still have some applications in Zend Framework, since we are using Piwik for our stats.

MySQL

MySQL 5.1 is our main database. The core base has 35 tables and more than 60 million rows. It is run on two master / slave machines (48Gigs of ram and 32 CPUs).

S3

Our file server used to be powered by NFS v3 to share logos, avatars and thumbshots, but we recently moved and mirrored our 200Gig of of assets into several Amazon S3 locations. Although we use Amazon CloudFront, we also publish some of our files through the level3 CDN.

Xen

Xen enables the virtualization of our “Fetch” software, which is running on 16 virtual machines. Fetch is managing the creation of thumbnails and the preparation of the pre-loading of the web pages when browsing a pearltree. Today, we can process 10.000 URLs / hr. We can then process our whole thumbshot basis in 15 days.

Bash, Python

The administration of our servers is mostly done with scripts. These are written in bash and python. We also use python in software that detects the URLs that can’t be viewed within an iFrame.

 

Backend

Bugzilla

We are using Bugzilla to define the priority of bugs and to follow their resolution.
Here’s the state of our bug list:

Subversion

Our whole code has been written in versions and is split in branches thanks to SVN. Through today we have made more than 23.000 commits.

 

Production architecture

Here’s a simplified version of our current architecture. We also have a pre-production and a development architectures that are similar to this one. UPDATE Sept. 2011 : The following sketch does not really reflect our current architecture as we have moved some services into the Cloud during the past several months.

17 Responses to “Technologies @ Pearltrees”

  1. Hello, I am somewhat of a web developer, what I mean is I just got into developing for web. I love the app it’s a pretty neat way to access informations on my interests. I have seen something similar to what looks like the peel tree on other sites. Maybe this is a rookie question, and I probably go through the FAQ in more detail. Can this be used on your or a clients website as a way to reference your little data trees and maybe display your content dynamically. Either way I love the app and will love more if it becomes useful as a developer.

    Thankyou for your time and hard work.

    Adam

  2. Adam, so glad you’re enjoying Pearltrees! The answer to your question is yes! You can get the embed code for a pearltree and add it to many different kinds of content management systems and blogs. Just look for the characters like this”" in the display window and there you will find the embed code for that pearltree being displayed. Simply copy this code and then paste it into your blog or website editor in HTML mode. This will display a pearltree on that site. And as you hoped, these pearltrees are completely dynamic. Any changes you may make to your pearltree will immediately be reflected in every embed.

  3. Hi Admin

    I am very interesting in the logic you use to arrange the pearls in the pearl tree. I am developing a small application that draws tree structures. Just to help me see visually the data inside my database. (The database design, is so that, the data is stored in a tree like structures)

    At the moment I have a very crud way of drawing out the tree, but I love the way pearl tree does it.

    Is there some where I can explorer the logic how pearl tree draws it out?

  4. Dear Admin.

    I am working at some kinda IT company in Korea. I am looking for new business model. While researching, pearltrees.com looks awesome, really cool. Do you have any plan to serve it as Korean, or any other languages??

    I am not a tech guy, but in my point of view as end-user, your GUI is very cool and love your business.

    Thank for your time.

  5. Thanks for your comment.
    We’ll translate Pearltrees in other languages at some point later :)

  6. Hello, I subscribed already with my iphone, ans don t know how
    transfer my pearltrees on my iphone with my computer
    thanks for answer

  7. je me suis inscrite sur pearltrees avec mon iphone et ne sais pas comment transférer avec mon ordinateur
    merci de votre reponse
    bien a vous
    mcc

  8. Bonjour,
    Il suffit de vous connecter sur votre compte pour retrouver votre pearltree à l’identique, sur ordinateur et iPhone :)

  9. Hello,

    I was wondering if Pearltree is readable by word recognition software or mobile devices (iPad)?

  10. Hello,
    You can find Pearltrees on your mobile device : iPad, iPhone or Android.
    Yet, it can be read by word recognition software for the moment.

  11. How dO I delete this account??? Thank YOu

  12. Hi,
    Go to http://www.pearltrees.com from a computer and log into your account
    Open the top right menu
    Chose settings
    Chose “delete your account”

    Kind regards

  13. Click Here Now for 4 Free Subliminal Messages Video Downloads
    (valued at $160). It was a box office success and put the film
    as one of the highest grossing animated films of Disney.
    They are iconic members of the Disney cast and no
    collection is complete without them.

  14. No matter if some one searches for his vital thing, so he/she desires to
    be available that in detail, thus that thing is maintained over here.

  15. Hurrah, that’s what I was exploring for, what a data!

    present here at this web site, thanks admin of this site.

  16. Oh my goodness! Impressive article dude! Thank you, However I am going through difficulties with your
    RSS. I don’t know why I am unable to subscribe to it. Is there anybody getting similar RSS issues?
    Anyone who knows the solution can you kindly respond? Thanx!!

  17. Superb post but I was wanting to know if you could write a litte more on this
    subject? I’d be very grateful if you could elaborate a little bit more.
    Many thanks!

Leave a Reply