Monday, December 10, 2007

my new favorite company - palantir

palantir is one of my new favorite companies. not only do they say the right things, i actually like what they do. first, here is what they say.

  • Your Are Brilliant, We are Hiring
  • We stand for elegance, beauty, and ease of use in industries that have traditionally been underserved in all these areas.
  • Analysis is fundamentally collaborative. When looking for signals in gigabytes of noise, isolation is a recipe for missed discoveries. An organization that can share the right information between the right people is far more intelligent than the sum of its parts.
  • we live and breathe good software. Even as we tackle problems from the domains of finance and government, we are driven by our roots in technology. We are top technical minds who get things done.
  • It's also true of our people - one brilliant coworker is worth ten above average ones.
  • Change the world through new ways of exploring information.
  • Brilliant, social teammates who are some of the smartest people you'll meet anywhere.
  • Flat, meritocratic structure means the best idea wins. Make a big difference on your first day.
  • Stock options in a company with real revenue and tremendous growth potential.
  • Beautifully engineered products that our customers love.
  • Well-funded startup environment with full salary and benefits.

  • wow, that is some cool statements. imagine if they actually believe and do those things. anyway, palantir is pretty cool.

    now on to the thing that i beginning to love about them. check out this blog post: Palantir: so what is it you guys do?. there is too much in the blog post to summarize. but here are some interesting points.

  • we specialize in analysis.
  • analysis is everything necessary to extract insight from information
  • the analyst (human) should be in control.
  • summarize large data sets (machine learning, statistics)
  • visualize large data sets
  • iterate rapidly - ask question, get answer, ask a variant question
  • collaborate with other analysts effortlessly
  • hardest part is data modeling, figuring out what data type are relevant to a domain.
  • Data modeling, data summarization, and data visualization are the core disciplines for approaching large data sets. Human-driven queries, rapid iteration, and collaboration are multipliers, taking the power unlocked by the core disciplines to the next level. When these pieces are brought together in a coherent system, the result is in an analysis platform both very generic and very powerful.
  • welcome to the future of analysis.

    WOW! this is the kind of stuff i have been thinking about for hackystat. we need some palantir approach to our hackystat analyses. this realization of sophistication is what makes analyses useful. information (the data) is meaningless if we don't understand it. to me, palantir is all about making meaning. that is a cool concept. they are insight providers.

    so... i think hackystat could use several things:
  • interoperate between levels of information
  • summarize large datasets, but also provide fine grain details when needed
  • make connections to others in the project team.
  • provide feedback
  • provide context
  • data mine to help find useful information
  • iterate over analyses and reports
  • share charts, comments about charts, usefulness context,
  • publish reports based on analyses
  • push out awareness
  • etc, etc.

    i like the idea of a palantir approach to hackystat. hackystat provides information. a palantir approach to the analysis provides meaning. simple charts and tables or even fancy charts and tables just doesn't cut it.

    palantir has an approach, i wonder if it will work. if it does, it will be the future of analysis.

    Philip Johnson said...

    Hi Aaron,

    I agree with you that the Palantir approach is what Hackystat needs. Indeed, I feel I already view the Hackystat "mission" from a similar perspective with respect to both the importance of analysis and the importance of people.

    My question to you is: so where are we failing to replicate the "Palantir vibe"? What should we be doing differently? How can we change our practices to get even better?

    aaron said...

    there might have been a little confusion in my posting. i think we (csdl and hackystat dev team) have the palantir vibe from an organizational/motivation standpoint.

    i was just merely stating how they seem to value the right things in terms of an organization mission statement. i mentioned this for all of the other companies out there. so, the upper posting of this blog doesn't really apply to csdl. hm... wait a minute, it could always apply to csdl. and that should be our continuing quest to get better as an organization. as of right now, i think we are providing that vibe, but we should be always be evaluating whether we are or we aren't.

    most of my blog post was to direct the readers to what they are doing in terms of analysis, and how palantir thinks about analysis.
    looking at what they do in the analysis area and thinking about hackystat, i think we eventually want to get to making user interfaces that make exploring the data much easier. for example, i totally love the different levels of abstraction that we have, sensor data, daily project data, telemetry, etc. but we are bounded by the implementations of that. for example, it is impossible to seamlessly switch between or drill down into various levels of information. or we don't have the capabilities to automatically generate trends based on the data we are currently looking at. instead, we have to switch contexts and "get out of the sensor data viewer and go to the telemetry viewer" if we want to see trends. in other words the actions of that analyst is bounded by the implementation of the separate views.

    i think there are a lot of other examples like that. in my opinion, once we finish up the measurement side of the framework, there are many opportunities to revolutionize user interfaces for exploration and collaboration. it is a hard task. but it seems really cool.

    i personally, think we should have an integrated killer user interface. i like how the UIs are separated now, but i think that serves just a portion of what we want to do. eventually, i think we should have a big webapp again, but this time more integrated based on tasks not just analyses.