For the last year or so I’ve been working on building a software application to help marketers allocate their marketing spend. This software is statistics and data-science powered and my partner and I have spent more hours than I’d like to admit struggling to squash bugs, achieve model convergence, and generally answer the question “why on earth could that be happening?” In this post I’ll discuss the history of the lab book and how it’s used generally before discussing how to use it for data science and software engineering projects and providing an example lab book template.

A Culture of Partnership During my time leading an analytics and data science team, I spent a lot of time thinking about how an ideal analytics team should operate – how the team should work together, how the team should prioritize their work, and how the team can most effectively partner with the broader organization to generate business value. I believe that for an analytics team to be effective, the team must develop a strong culture of partnership in order to actually drive business value.

The first data hires at an early stage startup face numerous challenges — an infrastructure built to run the business but not analyze it, an organization hungry for information without a process for requesting and prioritizing it, and little documentation on how anything is done. What should they do first?

Agile software engineering practices have become the standard work management tool for modern software development teams. Are these techniques applicable to analytics, or is the nature of research prohibitively distinct from the nature of engineering? In this post I discuss some adjustments to the scrum methodology to make the process work better for Analytics and Data Science teams.

This is part 2 of my 3 part exploration of the following question: are Agile engineering practices applicable to analytics, or is the nature of research prohibitively distinct from the nature of engineering? For the agile fans, in part 1 I gave an intro to agile and talked through what I like about the scrum development process for analytics. For the agile nay-sayers, in this post I explore the elements of agile that do not work particularly well with Analytics (issues range from annoyance to downright incompatibility).

Agile software engineering practices have become the standard work management tool for modern software development teams. Are these techniques applicable to analytics, or is the nature of research prohibitively distinct from the nature of engineering? In this post I am going to explore some of the pros of using a scrum-like work management process in analytics.