September 13, 2022

By: Krishna Mohan Rao Kaluri, Associate Director - Data Engineering, LTIMindtree

A deep awareness of the data management life cycle, exceptional project management skills, and a solid grasp of benefits management are essential for effectively managing and executing large/complex data-centric programs. The road to success can be filled with potholes if such programs are managed without prior experience with data management or if one assumes that a project manager without any data experience can successfully deliver a “data program.”

A data-centric program is one where data plays an important part in the success of a program. Examples of data-centric programs are –

• Data Platform Modernization: Move legacy/on-premises applications, workloads, data lakes, lake houses, data warehouses, and data marts to cloud platforms.
• Building platforms/applications that rely on the consolidation of disparate types of data from multiple source systems that need harmonization across dimensions
• Data lifecycle management projects
• Data-centric products
• BI/Analytics, dashboards, reporting programs
• Predictive modeling/Data science programs

This article sheds light on how to become a successful data program manager (DPM).
1. Data strategy and data governance: A Data Program Manager (DPM) should ensure that the program charter is in line with the organizational data strategy and roadmap for the future. Organizations that have well-defined data governance standards and processes and a clear roadmap for the future often see a high success rate in their data programs. The data strategy, data maturity roadmap, and data governance policies should be referred to when writing or evaluating a data-centric program charter. If organizations do not have the maturity to have a data strategy, then this program might be the best opportunity they have to create one. Hence the DPM should plan a small budget for creating a data strategy alongside this program (two birds in one shot).

2. Architecture: As a DPM, involve the enterprise data architecture teams to review a project charter and certify the program’s alignment to the laid-out strategy. This will save unpleasant surprises that you do not expect. Assess the data volumes that your program will have to create, distribute, ingest, process, or store. This will help in technical decisions, design, performance threshold planning, environment sizing, and multiple other areas that depend on the data architecture and data volumes. Most data-centric programs today revolve around cloud platforms or cloud data warehouses. Hence a DPM should have a firm understanding of cloud offerings, data services within cloud platforms, cloud DWHs, data lakes, data mesh concepts, DevOps concepts, and cloud cost calculations.

3. Design: When defining or recommending a design/solution or a data model, the DPM needs to ensure that his/her architects do not present just one choice to the decision board. For the best decision-making, it is always good to present more than one choice, with a recommendation to allow the decision-making board to evaluate and select the best solution based on the technical, schedule, budget, and other relevant aspects.

4. Discovery: Evaluate if the data program you are taking up needs a preliminary discovery phase. This phase allows uncovering a lot of unknowns hidden behind the scenes and minimizes risks to the development and testing phases of the project. In fact, a discovery phase done prior to the requirements phase will allow the requirement gathering to be bi-directional between the discovery team and the end user.

5. Data integration: Data integration is key to the success of any data program. The DPM should have a thorough understanding of various types of data sources and the issues they bring, internal and external integration points and limitations, ETL and transformation approach, batch vs. near-time vs. real-time nature of data integration needs, etc.

6. Data quality: Poor data quality can adversely impact a data program; hence it is imperative to assess this quality of data in the context/scope of the project. Data profiling should be conducted in the first stages of the project to identify the accuracy, completeness, consistency, relevance, comprehensiveness, granularity, homogeneity, redundancy, precision, etc., of data. A data-centric project will certainly fail or prove extremely unviable to re-engineer if the underlying data is of low quality.

7. Data masking: In cases where data masking is mandated, it is important to assess the risks that data masking would introduce to the data integrity, data quality/master data, and business rules of a data program. Testing with masked/obfuscated/scrambled data can lead to misleading results.

8. Approaches/ Standards: The program’s quality plan should call out specific approaches, strategies, and standards that the program should adhere to. E.g., Standards for architecture, design, coding, documentation, error handling, DevOps, etc.

9. Methodology: Every DPM should have an excellent grip over the project delivery methodologies. It is of utmost importance to understand how to design a program in an agile methodology to deliver measurable objectives and value, increased business visibility, and quick rollouts. Agile methodology is best suited for transformation programs where change is constant.

10. Estimations: As a DPM, you should evaluate an extremely clear view of the estimates of the program, especially if the estimates were done by another person/team, or entity. As you get more clarity on the project, the first order of business is to assess if the program can be delivered within the approved costs and estimates. Every single inventory and estimation assumption should be at your fingertips to catch scope creep.

11. Project charter and prioritization: Before starting a new program, it is the responsibility of the DPM to know the program’s priority within the portfolio so that the right resources are allocated to deliver the desired “value” or “outcome.”

12. Stakeholders:

• Stakeholder management gets neglected under the assumption that the daily stand-ups, weekly status reports, and periodic steering committee meetings take care of it. It does not, and it is in the DPM’s interest to have a stakeholder management approach.
• Spend time finding the stakeholders and the respective managing style that you should apply. (Refer to PMI for the stakeholder management styles).
• Ensure a minimum 20 percent FTE involvement of a business owner/business change manager to be involved in a data-centric program so that organizational/business changes are managed right from the start through handover.
• Ensure that the program’s business change manager is not a newbie to the program landscape and does not lack the deep SME knowledge needed to influence the user community.
• Expect conflict and be prepared to deal with difficult stakeholders with unachievable expectations/deadlines or tangential priorities. Map out a proper stakeholder who can influence these types of stakeholders

13. Planning and monitoring:

• Produce a plan with high business involvement and involved technical teams; ensure that the jargon used in the plan is clear and accepted by everyone. Providing a walk-through of the program plan on a page (PoaP) on a periodic basis is key.
• Validate assumptions on which the program has started, else invalidated assumptions might creep into the project and pose a risk/issue, eventually stalling the program.
• Transformation programs inherently involve unexpected changes. Ensure there is an element of progressive re-planning every week/fortnight and a provision or threshold to accept the changes is predefined.
• When delivering a long-duration program, you will need to manage the pace of the work by juggling the priorities and projects to ensure a measurable output gets delivered regularly. Avoid long phases of the SDLC at all costs.
• Do not create a megaproject plan capturing the lowest level of details in one plan. Remember, this is a program, so there have to be multiple tracks, and each project/track should have a detailed project plan owned and managed by the project manager or the track lead for the track. A program manager needs to focus on the rolled-up view or plan-on-a-page view for the entire duration of the project.
• Most programs fail because of the assumption that management of risks and issues is the responsibility of the program manager *only*; it is not; hence empowering your teams to find problems with mitigation to resolve is important.

14. Collaboration

• Teams and stakeholders have become increasingly remote, so make sure that there are good collaboration tools set up for the program.
• Communicating with internal teams is as important as communicating with business stakeholders. Keep spontaneous communication to a minimum; your goal should be to plan all communication.
• Create a robust organizational structure that outlines a clear chain of command with responsibilities, boundaries, and authority. Leave no scope for interpretation. Every team member should be accountable for their responsibilities.
• Micromanagement demoralizes your team. Empower your team by creating an environment of trust and transparency.
• It is important to delegate. A solid organizational structure allows you to delegate a workload that frees up some of your time. It not only allows your reportees to learn and get ready for the next level, but it will also allow you to keep 25 percent of your time free to manage unexpected issues/situations.

15. Change management

• Spend time defining tolerances between the C-suite decision-making authorities of scope, cost, time, and quality.
• Do not just include a statement in your program definition that says, after the document’s approval, changes are governed by change management procedures. It is important to include a usable process for this and get buy-in from the stakeholders to ensure the process works.

16. Testing

• Engage with the end-users right from the beginning (design, development, testing, demo phases). Data programs need a show and tell of the current vs. future state of data structures, to avoid surprises during the last phase of testing.
• Never assume that defects will be found and resolved in the testing phase. Most defects should be caught during the earlier phases of testing (unit, system, and integration testing phases with the use of functionally viable data).
• Develop a quality process where engineers provide evidence of unit testing and hold dedicated sessions with the BAs and architects to ensure the solution developed is as accurate as possible.
• Always ensure that a clean test cycle is baked into the overall program plan at the beginning of the program. This clean testing/regression cycle should begin with almost zero bugs. This will go a long way in ensuring massive retest and fixing efforts are not needed in the last stages of the program, risking the deadline.
• If it applies, always ensure that E2E testing is conducted as early in the project phases as possible. This will require considerable data preparation; hence, never assume that source data is readily available for testing; it never is!
• Never neglect the infrastructure needed to conduct development, SIT, E2E, performance, parallel, and UAT tests with a high volume of viable data.

17. Signoffs: Do not give up on getting your deliverables signed off promptly. If the approvals are not provided, try to obtain a conditional signoff to ensure progress is not hampered due to a delay in formal sign-off.