Development Category

Winners of the WhitePages.com Developer API Contest

Four months ago we launched the WhitePages.com Developer API to the general public and at the time we were going to be excited if we had a hundred active developers using the API within the first year. I can proudly report that we have greatly exceeded our initial expectations.

There have been conversations springing up on other online forums, people requesting it be integrated into their favorite application and over five hundred active developers. One fifth of those have queried our API over a thousand times and many of those have exceeded ten thousand queries.  Several small search engines have integrated our API into their results and two developers have launched Applications into the Apple iPhone store.

In conjunction with the launch of the API we announced a contest for the best iPhone application and the best social networking application to utilize the API. I’d like to give my thanks to all those developers who entered the contest. We saw some really great entries but alas there can be only two winners of the MacBook Pro’s we are awarding.

Adam Leonard of Caffeinated Cocoa won first place for his iPhone application ‘People’. ‘People’ is an excellent straight implementation of our services on the iPhone and allows people to save or merge pertinent results directly with their phones address book.

Trevor Hall took first in the social networking category for his facebook.com application ‘Visual Directory’. ‘Visual Directory’ encourages a real world connection with your friends by finding and mapping likely matches and providing contact information.

Coming in second were Philippe Furlan with an integration of the reverse_phone API into his ‘SearchQuest GPS’ iPhone application and Bryce Baril of Market Outsider for PartyBot.

You can see more about the contest at developer.whitepages.com or via this press release

This posting is also cross posted to our main blog.

Posted by Dan Sabath on August 6, 2008 – 10:28 am in API, Development | 0 Comments

3703 sites, 14 days to test, 4 people to do it

Here at whitepages.com, our Quality Assurance team is divided into two parts.  We have a front end team and a back end team.  While, as a group, our responsibility is to ensure that every product we ship is to the highest quality possible, our groups differ greatly in the tools we use for testing and the way we tackle our most difficult challenges.  This post is about the front end testing processes we use at WhitePages.com. 

The Challenge

When we bring in interview candidates for our team, it can be difficult to explain exactly what we do in our day to day work.  Sure, we have exciting projects that ship nearly every release, we have partner sites that require regular maintenance and coordinated releases (sometimes in the wee hours of the morning), and we have an endless backlog of minor bugs that we try to cram into any remaining room in the release cycle, but our job is more than that.

Whitepages.com prides itself on uptime, agility and an excellent user experience.  Those three goals affect our day to day work far more than anything we could fit into a given release. We power people search on over 3700 websites ranging from sites that get just a few hits a day to others that bring in millions of hits per day.

The challenge to our team of four front end QA engineers is to cover as much functionality as possible across as many sites as possible every release cycle to ensure we deliver a reliable and consistent experience to our users after every rollout. 

The key to achieving maximum coverage is a mix of manual tests to test new functionality and the critical use cases, and automated tests to check for regressions and code changes that affect a large number of our sites.  The rest of this post is about some of the techniques/tools we use in our day-to-day testing.

The Solutions

Manual Testing

Over the years, we’ve found the best way to do the basic bug verification for our standard releases is through manual testing.  It’s the quickest way to try out the functionality from the user’s perspective and make sure it does what its supposed to do.  The exception here is for changes that are in common code that affect all sites, in those cases we’ll check our top tier of sites manually then usually write up some quick tests using Perl or Selenium to test the changes on the rest of the impacted sites.  Anyone who has worked on front-end web apps knows that manual testing can be a nightmare though.  The matrix of operating system vs. browser vs. resolution, etc can make for a mind-boggling number of test combinations.

Browser usage by our users breaks down something like this:

  • Internet Explorer 7 – 45%
  • Internet Explorer 6 – 40%
  • Firefox 2.0/3.0 – 10%
  • Safari – 1.5%
  • Other – 3.5%

Given that, we tend to do about 80% of our manual testing in IE.  Unfortunately, debugging tools in IE aren’t so great.  We tend to rely heavily on the debugging tools/extensions in Firefox such as Firebug, LiveHTTP Headers and Web Developer.  Once we’ve debugged the issue and have a fix, we do our final testing and verification using IE.

Each release we also have a suite of manual tests we run to cover critical functionality.  While the manual suite is a bit tedious to execute week in and week out, we’ve found the manual high-level pass to be a critical step to wrapping up our testing for the release. 

Automated Testing

WhitePages.com is a predominately a Perl shop.  The code that powers our site is written in Perl and a lot of the test automation code is written in Perl as well.  Everyone on our front-end QA team knows Perl.  We all write the tests, we all execute the tests, we all debug the tests.  When it comes to writing front end automation tests in Perl, CPAN is our friend:

LWP is a great way to write some quick tests to grab HTML source, scan through it for content and return a pass/fail based on the presence of a given bit of code or text. 

WWW::Mechanize makes form handling a breeze and has some excellent built-in functions for checking links and analyzing image elements (check the URLs, check ALT text, etc). 

XML::Simple provides a quick way to pull down basic XML and parse through it for a given value without a lot of overhead.

All of the modules above basically request the source of the page and use that to determine whether a given test passes or fails.  Often times, that isn’t enough for us.  Our users don’t look at the source, our users look at the rendered page.  Because of this, we often need to test things in an automated fashion from the user’s perspective. 

For automation through the browser, Selenium is what we’ve found works best for us.  Its open source, it allows you to test in multiple browsers and, the best part, you can drive it through a Perl script using the WWW::Selenium module on CPAN.  We’re working on some very interesting projects using Selenium including a model-based web testing framework to test our rendered UI and a project leveraging Selenium and the User Agent Switcher extension in Firefox to test our Mobile Site against a wide number of mobile user agents.

Even with the best of tools and techniques, there are some great challenges associated with testing a top 50 website that powers thousands of partner sites and pushes new code to production every few weeks.  We’re always looking for new and better ways of testing our products and for sharp people who want to join our team.  

 
 
 
 
 

 

Posted by tpearl on July 21, 2008 – 4:31 pm in Development | 0 Comments

Capturing Perceived Render Time & an upcoming Jiffy feature

While attending the panel on web performance monitoring, the topic of measuring the user’s perception of load time came up, and Steve Souders mentioned that he hadn’t seen anyone write about this - since we’ve done some work in this area, we figured we’d write in more detail about what we’ve done. As a bonus, here’s some information about an upcoming Jiffy feature.

At WhitePages.com we learned long ago that the page was usable long before it completed rendering. This caused us to search for the Holy Grail of perceived rendered time, as Jack has previously mentioned. Unfortunately, this is easier seen with the eyes than it is measured programatically.

Our first attempt was to capture the DOMReady event. That event is commonly defined as the point at which the HTML has been completely downloaded although all of the assets referenced within that HTML ( i.e. script files, image files, XHR requests, etc. ) are still being requested and loaded. Many of the JavaScript libraries have this event as one that can be bound to and it seemed simple to just attach a Jiffy measure point to get the elapsed time to DOMReady. The issue is that not all browsers really support that point in time,  and some just approximate that event. Normally this is not a problem: the event is primarily used to attach functionality that requires the DOM be complete before they are attempted. For measuring purposes, though, it gives us different reference points across the different browsers that are not truly equivalent points in time.

This lead us to dig further into what exactly we were trying to measure. We didn’t really want to know when the DOM was ready, but when the user could interact with the page. On WhitePages.com, the primary use of the page is a search: the first thing most users do in start typing inside the search form. The best way to find when the user could use the search form was to emulate the behavior by clicking into the search field. If we can capture when the user can ‘focus’ on the search form fields, we have found the point at which a user can interact with the page.

This is a good general rule of thumb, which Scott discussed on the Measuring Performance panel - don’t depend on capturing a browser event to approximate interactivity; instead, try to capture that interactivity.

Since DOM objects like input boxes are rendered and accessible almost instantly, we decided that we would create a function for capturing events that can be attached to objects like this. It would allow us to see when the input field is focusable which would be when the user could start typing into the field and search.


<html>
.......
<input type="text" id="person_search"></input>
<script type="Javascript">
Jiffy.capture("person_search", "focus", "searchDone");
</script>
.......
</html>

Now, those who’ve read the Jiffy code have noticed that there is no “Capture” method yet - we’re still testing it on our site, both for the focus event and more generally. One thing that makes this tricky is that ads occasionally attempt to capture focus (shocking!).

Capture is a great addition to the library to allow you to attach to events, though, and you can see how it would work by its parameters:

1. The ID of the element we want to attach and execute an event on.
2. The Event we want to attach the Jiffy measure call to.
3. The measure name sent to the logger.

Today this capture only works for focus but going forward we would like to be able to do this for other events such as load of an image tag or YUI’s version of content loaded.

Posted by Ben Maldonado on June 24, 2008 – 7:39 pm in Development, Velocity08 | 0 Comments

From the audience at Velocity2008: David Slays Goliath

The parade of morning keynotes and product introductions included some heavy hitters, namely Keynote and Google, and some lightweights, WhitePages.com. Taking a back seat to these guys is usually fine with us as we stay out of the line of fire and can make mistakes with very few people actually noticing. We also learn a lot from the bigboys and tend to follow their lead. After all, these are the industry heavies and can attract and hire more smart people and buy more infrastructure then we could ever hope to acquire.

Keynote demoed a free product today called KITE (Keynote Interest Testing Environment) which allows developers to instrument a workflow or click-stream in a web application and benchmark it from 3 locations in the US to gauge performance and user experience. For a free service it is pretty good, and the audience seemed somewhat receptive to the idea that it’s okay to send your performance data to Keynote, but someone behind me muttered, “how much does it cost for more than 3 datacenters, I wonder?”

Then Scott took the stage and announced Jiffy, which is basically the same product except you get performance telemetry from all the streams you care to implement from all of your users a few seconds after they happens. Oh, ours is free too and we give you the source.

A little guy with a slingshot (a good idea) and a few round rocks (some wicked free code) can do more than break a few windows …

Scott Ruthfield Announces Jiffy

Posted by Jack Valko on June 23, 2008 – 12:26 pm in Conferences, Development, Hacking, Operations, Velocity08 | 0 Comments

Introducing Jiffy: Real-World Browser Instrumentation and Reporting

During 2007, we shipped a lot of new features on WhitePages.com and other sites in the network. While each of these improved the functionality and usefulness of our service for our customers, we weren’t doing a great job tracking the performance impact of these changes – and so our site was getting slower. Performance is a feature too and we had ignored it.

So we grabbed Firebug and started loading our pages over and over again, looking for patterns. This anecdotal foray just sent us on wild chases, so we backed up and looked for a system that would help us see what our customers were seeing, and allow us to measure every little thing – when individual third-party components loaded, how long it took the search forms to render, etc.

We didn’t find one. We found some third party tools from the usual suspects that sell performance monitoring services – we even implemented one of them – but we found that they weren’t flexible enough to give us the data we needed, slowed down our site, and weren’t real-time enough for our taste.

So for Hack Week, Don & Jack hacked together a project that would log how long it took to render each page in the proxy logs, and then report on the performance. Later in the year, Ben came on board and added the ability to capture individual moments on the page and write those to the proxy log as well, Devin designed the database schemas and rollups, John and Travis made sure it worked, and I… did some things. So all told, we built this thing that we now use that tells us how long some things take to happen. No single part of this system is rocket surgery, but the entire combination is truly something that we haven’t found in the market from anyone – paid, free, self-hosted or service-hosted. We’re proud of the work and already see benefits from it.

Additionally, late last year, we were discussing how we wanted to give back to the open source community: as a LAMPP (Linux, Apache, MySQL, PostgreSQL, & Perl) shop, open source systems have played a core role in our company’s technology stack. Individuals on the WhitePages team have occasionally contributed to projects, but we didn’t have anything significant that we as a company brought to the table.

So today, we’re releasing that performance toolset as an open source project, under the Apache 2.0 license, with the genuine hope that other web publishers get benefit out of the ability to gather real-world measurements from all of their users about the timings of individual aspects of their pages. Jiffy is the name for the toolset. There’s a whole other blog post about the naming process, but a jiffy is a standard term for a small unit of time, and that’s what we’re helping you measure.

We announced the public availability of Jiffy this morning at the O’Reilly Velocity Conference, with a Plenary session following Bill Coleman (the B in BEA Systems – I think of myself as the R in WhitePages.com) and a new release from Keynote Systems. The code is linked from code.whitepages.com (also a new site for us) - the slides are there as well, the video will be available soon. We’ll have more to say about Jiffy in the upcoming days, but in short – take a look, let us know if you have questions (you can use the Jiffy Google Group so everybody can see them, but of course we’ll be here too), and we look forward to seeing how others use Jiffy for their sites!

Posted by Scott Ruthfield on June 23, 2008 – 9:19 am in Conferences, Development, Operations, Velocity08 | 0 Comments