attack of the twittering ian

i just happened to stumble upon twitter's explore blocks visual application. the idea is pretty simple and cool:

Explore your Twitter Block: Discover new people on your Twitter block by navigating through this animated three dimensional visualization of who follows whom.

here is what my twitter block looks like:

(the title is a joke... zoom into the picture. i'm just pointing out that ian twitters a lot, which i think is awesome).

anyway, so i thought that the twitter blocks was a pretty cool idea on how to explore and find new people. it doesn't work that great. we have a long way to go to make visualizing data useful and intuitive.

this is an interesting area, but i definitely need to learn more about it. here is one book that is pretty cool: Visualizing Data. here is a recent blog post from palantir tech.

we are definitely going to focus on this area a lot more in the coming months. stay tuned.

my new favorite company - palantir

palantir is one of my new favorite companies. not only do they say the right things, i actually like what they do. first, here is what they say.

Your Are Brilliant, We are Hiring

We stand for elegance, beauty, and ease of use in industries that have traditionally been underserved in all these areas.

Analysis is fundamentally collaborative. When looking for signals in gigabytes of noise, isolation is a recipe for missed discoveries. An organization that can share the right information between the right people is far more intelligent than the sum of its parts.

we live and breathe good software. Even as we tackle problems from the domains of finance and government, we are driven by our roots in technology. We are top technical minds who get things done.

It's also true of our people - one brilliant coworker is worth ten above average ones.

Change the world through new ways of exploring information.

Brilliant, social teammates who are some of the smartest people you'll meet anywhere.

Flat, meritocratic structure means the best idea wins. Make a big difference on your first day.

Stock options in a company with real revenue and tremendous growth potential.

Beautifully engineered products that our customers love.

Well-funded startup environment with full salary and benefits.

wow, that is some cool statements. imagine if they actually believe and do those things. anyway, palantir is pretty cool.

now on to the thing that i beginning to love about them. check out this blog post: Palantir: so what is it you guys do?. there is too much in the blog post to summarize. but here are some interesting points.

we specialize in analysis.

analysis is everything necessary to extract insight from information

the analyst (human) should be in control.

summarize large data sets (machine learning, statistics)

visualize large data sets

iterate rapidly - ask question, get answer, ask a variant question

collaborate with other analysts effortlessly

hardest part is data modeling, figuring out what data type are relevant to a domain.

Data modeling, data summarization, and data visualization are the core disciplines for approaching large data sets. Human-driven queries, rapid iteration, and collaboration are multipliers, taking the power unlocked by the core disciplines to the next level. When these pieces are brought together in a coherent system, the result is in an analysis platform both very generic and very powerful.

welcome to the future of analysis.


WOW! this is the kind of stuff i have been thinking about for hackystat. we need some palantir approach to our hackystat analyses. this realization of sophistication is what makes analyses useful. information (the data) is meaningless if we don't understand it. to me, palantir is all about making meaning. that is a cool concept. they are insight providers.

so... i think hackystat could use several things:

interoperate between levels of information

summarize large datasets, but also provide fine grain details when needed

make connections to others in the project team.

provide feedback

provide context

data mine to help find useful information

iterate over analyses and reports

share charts, comments about charts, usefulness context,

publish reports based on analyses

push out awareness

etc, etc.

i like the idea of a palantir approach to hackystat. hackystat provides information. a palantir approach to the analysis provides meaning. simple charts and tables or even fancy charts and tables just doesn't cut it.

palantir has an approach, i wonder if it will work. if it does, it will be the future of analysis.

is everyone frustrated?

i often get frustrated with technology. frustrated that people don't think of better use cases, additional use cases, and identify that special piece that could make something great. after a short period of frustration, i sit back and wonder why i just got so frustrated. i wonder why doesn't these things bother other people. i feel like the only one that realizes that this sucks and that it could be so much better. being frustrated with technology isn't good. its frustrating that its so frustrating.

well, come to think about it... i get frustrated a lot with a lot things. and yet i get over it so quickly.

learning how to run code reviews

yesterday, i sat in a fantastic code review moderated by austen. to some degree for the past few times we have been working on his ability to moderate code reviews. let me just say that it is really working out very well.

in my opinion, moderating code reviews is definitely a skill. if you don't think so, then i wonder if you have moderated an effective code review. to be the moderator you need to make sure that the time is spent effectively. also, the most important thing is that you make sure that everyone is learning the whats, whys, wheres, and hows of what is going on. another thing to do is to make sure we don't focus on superficial issues and identify where we can make tangent discussions about design. also, leading the discussion in a positive direction is very important. moderators that just say, "needs fixing, follow the coding standard, moving on" are really not doing an effective job.

for this specific code review we spent a while explaining why coding standards are standards by identifying why specific "rules of thumb" have been created. we also, were able to shift the discussion (almost accidentally) to a larger more important design discussion (a totally awesome design discussion).

anyway... i'm fairly confident that austen has the ability to lead effective code reviews. i really think that it is an invaluable skill for a software engineer that takes practice. this skill combines a lot of the soft skills that i'm talking about.

in my opinion, it takes a deep understanding of not only knowing how to design software but also how to help teach that understanding to other people. that sort of skill separates austen from others.