<p>I had a question for you specifically, as I believe it relates strongly to the field you work in.</p>
<p>I've been looking at careers and specific jobs that have to do with things like optimization, statistical arbitrage, data analytics, data algorithms, data mining, statistical modeling, etc... </p>
<p>Now I don't actually know very much about what those activities entail, but from what I've read about them they sound like really interesting, perhaps exciting things to work with (at least for a numbers guy like myself).</p>
<p>My question for you is what is it actually like to work with advanced statistical methods and do computer modeling? For me, I love analyzing numbers and data sets and I have been doing it in my head my entire life since I was a child. I simply have a student version of minitab for an entry-level stats class and I just love all the analytical tools on there. For me it's a blast.</p>
<p>I just don't know if I would like to be doing it 40-60 hours a week the rest of my life. Is it a career that you find satisfying? Fun? Boring? Antisocial? </p>
<p>Any "insider" insight would be very appreciated.</p>
<p>Someone asked a question about data mining (I think in the Science thread) on why it has not “totally” taken off (I think it eventually will) and (to me) the biggest obstacle is that once the data warehouse is built (usually needed before data mining can begin), all of the funding is gone and/or the DW owners decide that the DW gives them enough data for now.</p>
<p>Now to answer you question…</p>
<p>Analyzing data and finding trends and creating associations using statistics is fun. This is where what we say “data turns into information”. You just hope that the project gets to that point where one receives all of this data to start looking into optimization, data mining and statistical data analysis. It’s great when you are actually writing code to simulate one of those hypothesis-testing formulas from some probability/statistics courses that you have taken.</p>
<p>Anti-social?..Oh no. When dealing with data and databases, you are constantly interacting with people because they need that data or the now “information” to do their jobs.</p>
<p>Let’s go like 2 or 3 levels below the complexity of data mining. Just a regular 'ole data architect like myself (I am more of a data architect than DBA) may have to design some aggregate views of data (called materialized views) so that a higher-up manager can use for their meeting. At their level, they don’t need to know the low-level or atomic transactional data. They need aggregated and summarized data.</p>
<p>Now that description is baby stuff compared to some data mining. With mining, we are drawing associations and trends that could be used for mission-critical or military decision-making. Oh by the way, with data mining, you are looking at MILLIONS and near BILLIONS of records. What I am trying to say is that the centerpiece of these organizations is DATA and how it is converted to INFORMATION. When you are involved in that, you are very much needed.</p>
<p>Thanks for the great response, Globaltraveler.</p>
<p>Just to piggy-back on the thread a little bit…</p>
<p>Do you think a CS background is sufficient for that sort of work? Or, do you really need more of an applied math/statistics background?</p>
<p>I’m a EE, but very interested in data analysis, operations research, etc. I’ve got a pretty good background in CS and would like to have the option of breaking into those sorts of fields after graduation. Should I be taking a lot of stats and applied math, or will just CS and engineering math do?</p>
<p>You can get the needed knowledge from just a few statistics electives. Most engineering majors have to take that combined “Probability & Statistics for Engineers” course which will not dive deep into either area for data analysis. I would suggest taking the Applied Statistics I,II because it will go into the various methods since there are several (Chi Square, Non-Parametric, etc). There are also the Mathematical Statistics I & II sequence but that will have a lot of theory also if you like theory. Me?..I am fine with the theorems being established I just want to apply it.</p>
<p>Now those additional statistics courses will just prepare you to learn data analysis ON THE JOB. The industry has a whole slew of applied methods that are not taught in academia. The extra courses just prepare you to learn the more advanced methods.</p>
<p>Having said that, many “cool sounding courses” are at the graduate level. In grad school, I had (not the “cool” ones) Applied Stats, Design of Experiments, Taguchi Methods and Statistical Quality Control. I always wanted to take the Data Analysis I & II courses but never got around to them. Oh, I did take a graduate Stochastic Processes course (a couple of years ago) and that was cool also.</p>
<p>The same with Operations Research. The first course teases you with trending analysis, linear programming, queuing theory but there are courses on EACH of those areas also. I also had a grad linear programming course. I took the LP and Stoc Proc courses in preparation for a contract that Boeing THOUGHT they were going to win…but didn’t :-(</p>
<p>So just extra courses is all you need.</p>
<p>I will throw this out here…I know Princeton and Cornell has a M.Eng degree composed of mostly O.R. courses. I was always into looking at engineering degrees that were composed of basically math courses.</p>
<p>I’ve been looking at Stanford’s Department of Management Science and Engineering, which looks pretty awesome. They have systems modeling & optimization, decision and risk analysis, probability and stochastic systems, and operations research. Does anyone know how it stacks up against Cornell and Princeton?</p>