A Data Scientist’s Daily Work 2 – Understanding Company Organizational Structures Before Job Hunting

Series Articles

〈A Data Scientist’s Daily Work 1 – Mining Business Value from Data and Code〉
〈A Data Scientist’s Daily Work 2 – Understanding Company Organizational Structures Before Job Hunting〉
〈A Data Scientist’s Daily Work 3 – Building Data Team Culture and Programming Standards〉

Organizational Structure is a Science

Since data scientists and data analytics departments haven’t been around for very long, people’s understanding of them still varies. Due to different core values and management philosophies across companies, data teams can exist in various forms. There are three common structures: independent departments, reporting to IT (Information Technology) or RD (Research & Development), and reporting to business requirement departments.

This article primarily discusses in-house data teams, commonly known as “Party A” in Taiwan, which could be brand owners, distributors, or manufacturers. We’re not talking about consulting or project-based companies, known as “Party B” in Taiwan, such as advertising agencies. Perhaps in the future, we can discuss the differences between Party A and Party B.

Structure A. Independent Department

An independent Data Team department is my favorite structure. This means the department brings together a group of data scientists or data analysts, and larger teams might even have dedicated Project Managers (PM). Without a PM, data scientists and analysts might need to wear multiple hats, handling cross-departmental communication and timeline management themselves. The main job of such data teams is to assist other departments. I like to think of it as a small company within the company, providing B2B data analytics services.

Structure B. Reporting to IT or RD

Let me first explain the difference between IT and RD. IT typically refers to departments responsible for system integration and server operations – more like infrastructure. RD, on the other hand, handles software development, such as app or web development. Most companies probably have IT departments, but not necessarily RD departments.

Due to these differences, even though they all involve programming, the code written by IT differs from RD, and RD differs from data analytics. Even though software engineers and data scientists both write Python, their work objectives and environments can be worlds apart. In other words, if your job title is data scientist but you report to IT or RD, and your manager lacks a data science background, they might not know how to lead such a team, what the team can accomplish, what resources it needs, or where the team might make mistakes.

Structure C. Reporting to Business Requirement Departments

Business requirement departments could be sales, marketing, or supply chain departments. When reporting to these departments, you’ll receive the least technical support, as you might be one of the few people with programming skills in the department. Moreover, business departments typically don’t hire too many analysts – one to two people is a common setup.

The following content will discuss several different scenarios and the pros and cons of these three structures in each situation. To avoid redundancy, I’ll refer to them as Structure A, Structure B, and Structure C.

Personal Career Development

In terms of data science technical skills, Structure A is best, Structure B is second, and Structure C is weakest. For industry knowledge, Structures A and B emphasize breadth, while Structure C emphasizes depth.

In Structures A and B, you have opportunities to collaborate with various teams, such as sales, marketing, and operations teams. This is especially valuable for workplace newcomers, allowing them to explore personal interests in real-world scenarios and develop second expertise beyond data science. With more people and projects in data teams, if you’re proactive, you’ll have better chances to choose projects and develop in directions that interest you, rather than working on projects with low achievement or visibility.

In Structure C, suppose you report to the marketing department. You might gain deep insights into how marketers operate commercially – how they plan campaigns, what metrics they care about, and what marketing techniques or tools they use. If you’re already certain your career path involves both data science and marketing, this could be a great direction for you.

Like-minded Colleagues

Everyone needs teammates and like-minded colleagues, not just for psychological validation, but also for having people who’ll stand with you in organizational structures and various small groups, and who can provide support when your workload explodes.

Data scientists need hybrid background knowledge – programming, statistics, business logic, machine learning models, and presentation skills. The job requires constantly switching between code and business thinking. Proactive analysts might even organize reading groups and study papers. All these characteristics show the mixed-blood DNA of data teams. Therefore, letting them flock together might be the best management approach – that’s Structure A.

In Structure B, you might still chat with colleagues about technical topics or share engineer memes. In Structure C, well, you might feel a bit lonely.

How to Choose Projects

Independent departments also have weaknesses. The first weakness is that independent data departments heavily depend on the business acumen and diplomatic skills of managers, team leaders, or PMs. Diplomatic skills include relationships with company executives and other departments. For example, data science projects can be initiated in two ways: supply-driven or demand-driven. The former is when you have a great solution and your manager pitches it to other departments. The latter is when business departments have pain points they can’t solve themselves, so they seek help.

In my experience, supply-driven approaches often fail, like vendors making cold calls – it’s hard to establish immediate cooperation, and there’s usually a long observation and evaluation period. Business departments always have established workflows that everyone follows peacefully. Data team intervention inevitably creates an adjustment period. Even if there are long-term positive impacts, short-term disruption might increase the business department’s workload. If the data department’s solution doesn’t hit the business core, it’s likely to fail. If failed cases spread within the company, other departments might keep their distance from the data team.

Conversely, when business departments have pain points and approach you, these situations can be divided into good projects and bad projects. Bad projects typically involve using SQL to extract data, then organizing it with R or Python into a decent report, exporting it, and that’s it. Such projects don’t showcase the data team’s value – frankly, the business department just needs someone to write code to extract data for them. If you’re performance-oriented, you’d naturally avoid such projects, which again tests the team leader’s diplomatic skills. From the business department’s perspective, they might genuinely lack channels, technical skills, or permissions to extract data, and this data might truly help improve their performance. Whether they’ll credit the data team for subsequent performance improvements is another matter entirely.

So what are good projects? Good projects are when business departments have pain points, know the data team can solve them, approach you for help, and the data team can demonstrate professional value in the project. The difficulty is that business departments need to understand what data teams can accomplish and be willing to propose collaboration. But ordinary people don’t understand prediction, regression, classification, or clustering – they might not even know what data you can access. This completely depends on the data team’s past achievements and the PM’s self-promotion and project management abilities.

For example, regardless of industry, inventory management is always crucial because inventory equals tied-up capital. Without inventory, you can’t do business. Whether order quantities are sufficient or excessive, whether inventory turnover is fast enough – these are all discussable topics. For relevant departments, inventory management is a pain point, but they might just shrug and say, “The system gives us suggested order quantities.” In such situations, business departments might not know data teams can help with inventory management analysis, or they know but modifying systems is too massive an undertaking. Without urgent necessity, they postpone it until executives notice and panic ensues.

In project selection scenarios, Structures A and B are more likely to reject bad projects but also less likely to proactively discover good projects due to distance from business departments. Structure C has the weakest ability to reject bad projects because business departments and you report to the same department and manager – they might say this project has departmental management approval.

Departmental Politics

In workplace ecosystems, departments that make money for the company usually have the most say. Data teams are essentially support departments – their results don’t directly generate value. Real benefits to the company only come after other departments take business actions. The result might be that data team performance links to company revenue, yet they’re not the ones actually executing business operations. This causes many data analytics projects to be cancelled before implementation due to lack of support from other departments. Even when projects succeed, credit attribution heavily relates to organizational structure.

For example, suppose the company held a major promotional campaign during Double 11. The data team’s task was selecting customers likely interested in the promotion for marketing to target with ads. If this campaign achieved very high revenue, it’s time to review campaign effectiveness.

In such campaigns, the lead might be sales or marketing departments, meaning they’ll likely report revenue and profits to executives. Sales might say, “Because we chose good products and gave deep enough discounts.” Marketing might say, “Because our copy resonated with consumers, our visual design was attractive, and product photos looked super appealing.” The data team might say, “Because I delivered attractive product information and promotions to the right consumers.” All these statements might be correct, but they can’t be called precise conclusions, like “Sales contributed 40% of revenue, marketing contributed 30%, and data contributed 30%,” because these conditions are hard to quantify objectively. Even if calculable, it would consume enormous time and manpower. In practice, few people carefully calculate these numbers, and even if you could, you might not convince others.

In this case, if you’re Structure C’s data team, the departmental manager will definitely mention their team’s efforts and effectiveness when reporting, using data power to strengthen credibility. Conversely, if you’re Structure A or B’s data team, whether anyone from your department gets invited to such meetings is unknown. The data team might work very hard with great results, but company executives might never know.

Your Data Might Offend People

One advantage of data-driven culture is digitizing company activities, thereby enabling scaling and transparency. Before implementing this culture, many company operations might rely on rules of thumb, with data application through manual Excel processing. This inevitably creates obstacles during digital transformation.

First, rules of thumb are indeed useful for quick business decisions, unlike data analysis which requires data collection and research. However, rules of thumb might contain personal subjective biases and life experiences, preventing comprehensive views or keeping up with market changes. When rules of thumb conflict significantly with data analysis, data scientists might receive feedback like, “Are you sure your analysis is correct? This differs greatly from our understanding.”

For example, I often hear similar comments in meetings: “Are you sure people aged 30-40 prefer Product A? I’m also that age, and I prefer Product B. I think this promotion should push Product B.” In such moments, I usually choose to smile and let it pass, because data analysis typically focuses on overall distribution, and each person is just one point on the distribution curve. How can you be sure you’re not an outlier in your demographic?

Second, there’s data acquisition and application methods. The same Excel report that colleagues might spend an hour organizing daily could, after implementing data analytics processes, require two weeks of initial development. Once developed, no manual work is needed – you can read automated reports when you arrive at work each day. This saves manpower, increases accuracy, and offers better scalability.

In both situations above, data culture introduction attempts to provide better solutions but might partially negate past workflows, even making workforce reduction a possible option. In this process, some people with weaker learning abilities might resist based on job security instincts, magnifying new methods’ shortcomings and immaturity, becoming opposition forces in digital transformation.

In this scenario, Structures A and B using data to reveal truth feels somewhat like an internal audit department. Structure C stands more on the same boat as business departments but might also create internal divisions by revealing truth.

Conclusion

Workplaces inevitably involve competing for resources, visibility, and pushing work around. When these situations occur, knowing what organizational structure you’ll be in and how many people stand with you can serve as job-hunting references. Besides pre-simulating future daily work, you can also preliminarily judge whether your personality suits the position.

For example, if you dislike attending numerous meetings and handling interpersonal matters, but the position requires you to act as PM, can you accept that? Or if you won’t have a data science background manager and must handle all professional issues yourself, likely requiring self-directed learning after work, would this be your ideal work-life balance?

This article covered a lot, with the basic assumption that data analytics can help companies grow. However, data analytics has an amplifier effect – it can amplify advantages and losses alike. How can you avoid mistakes on the data science path? See you in the next article.