Top 100 reports abound, and behind each one there is a good dose of personal bias and subjective selection.
As readers, we have no choice but to trust the experts’ analyses because we lack the tools and data to make comparative evaluations.
So, how reliable are, for example, “top 100” or “the best….” rankings? Alexander Lex, a postdoctoral researcher at the Harvard School of Engineering and Applied Sciences (SEAS), and colleagues created an open-source application they call LineUp that helps ordinary citizens like us make quick, easy judgments about rankings based on multiple attributes.
Lex says:
“It [LineUp] liberates people. Imagine if a magazine published a ranking of ‘best restaurants.’ With this tool, we don’t have to rely on the editors’ skewed or specific perceptions. Everybody on the Internet can go there and see what’s really in the data and what part is personal opinion.”
With LineUp, said to be the first dynamic visualization of its kind, users can assign weights to several parameters to create a custom ranking. For example, you may look at the raw data behind the best areas to live in and decide for yourself the relative importance of train services or good primary schools nearby.
Lex, along with colleagues, Hanspeter Pfister, An Wang, Nils Gehlenborg, Marc Streit and Samuel Gratzl, achieved the best paper award for LineUp at the IEEE information Visualization (InfoVis) conference in October 2013.
LineUp, an open-source visualization framework
LineUp forms part of Caleydo, an open-source visualization framework that visualizes biological pathways and genetic data.
Pfister explains:
“LineUp really was developed to address our need to understand the ranking of genes by mutation frequency and other clinical parameters in a group of patients. It is an ideal tool to create and visualize complex combined scores of bioinformatics algorithms.”
Lex adds “We started thinking about how we can make this easy for biologists to understand and how we can tell them what the most important parts of the dataset are.”
While initially designed for genetic research, the team has chosen to apply their technology to simpler and more familiar ranking problems, such as the top employers or the best places to live in.
Lex and team say LineUp provides a dynamic element to hitherto static analyses usually done on an Excel spreadsheet. The user is able to immediately include or ignore columns in a dataset by just dragging them into or out of a window. You can also have alternative weighting systems so they can be compared side-by-side.
Not all metrics contribute to an item’s rank identically. Higher values in certain metrics imply a superior rank, but not in every case. LineUp can invert a dataset, for example, a lower crime rate can correspond to a higher quality-of-life rank. The user is apple to rapidly apply and visualize the results of his or her intuitions.
It is useful for predictive applications
LineUp also has the potential to be used predictively, because the user can easily pose many “what if” scenarios.
So, for example, an automobile maker could devise a way to efficiently increase a car’s rank in a list of “best automobiles”, possibly by giving more weight to fuel efficiency over style in the design process.
Lex says “Essentially, it’s a tool to allow people to explore the complexity of reality.”