By: Sujith Gopalakrishnan, Managing Principal Architecture , Snowflake Business Unit

Today, most organizations with legacy data systems are focusing on modernizing their platforms, aiming to cultivate a data-driven culture, capitalize on their data assets, and embark on substantial data-focused initiatives. Building a new data platform from scratch using Snowflake is essential to enable innovative business capabilities, enhance customer satisfaction, gain insights, and unlock business value.

When organizations embark on building a data cloud platform using Snowflake, greenfield initiatives are usually enabled through either the ‘lift and shift’ approach or a process of reengineering, often referred to as ‘re-architecting.’

• Lift and shift: This approach is adopted when organizations prioritize swiftly migrating their existing legacy data, workloads, and functionalities to Snowflake as they are. This approach aims to address legacy data platform’s constraints on scalability and cost.
• Reengineering: On the data platform modernization journey, organizations prioritize integrating additional data sources, supporting new insights, or envisioning additional enterprise capabilities like data monetization and building new data initiatives. In such cases, building the data platform from scratch and opting for a reengineering exercise is imperative. This initiative primarily focuses on completely redesigning and developing new capabilities using Snowflake.

Reengineering initiatives could entail significant risks and challenges if not managed correctly, given the complexity of technology decisions, people, and processes involved.

Risks in Snowflake reengineering initiatives

1. Lack of expertise in managing a large-scale ecosystem

Reengineering initiatives often entail designing and building various data functionalities within and around Snowflake from the ground up. This involves finalizing the Snowflake architecture, creating a data model, integrating new data sources, building data transformation pipelines, ensuring data quality, governance, security, observability, DataOps, and enabling new consumption patterns, including BI reporting, data science, or building data apps. A lack of expertise in these multiple areas or improper prioritization of activities can lead to adverse effects.

2. Lack of a proper methodology and appropriate tools

The absence of a structured approach to navigating the nuances often results in delays, failures, and a lack of confidence in the initiative. Furthermore, the lack of methodology, frameworks, and accelerators can lead to a loss of focus and delays in the overall initiative.

3. Delayed time-to-market

Designing and building a new data platform involves multiple aspects that can potentially delay the program. Factors contributing to delays include the readiness of the business, the time required to finalize use cases and requirements, coordination across multiple teams, experimentation and evaluation of various solutions and tools, poor data quality, lack of ownership, and challenges in planning and prioritizing various modules. Without a structured approach, there may be delays in developing a Minimum Viable Product (MVP) and bringing the final data product to market.

4. Cost overrun

While accounting for new cloud infrastructure and licensing costs is implicit in a greenfield exercise, the lack of optimization on these services, long trial periods, failures, and delays in solution finalization can impact project timelines. The factors mentioned earlier that lead to delays in time-to-market can also result in extended personnel costs, further adding to the continued operational costs of legacy data platforms.

Key strategies for enabling Snowflake greenfield reengineering implementations

1. Rely on a proven methodology

Adopt a structured approach for the reengineering exercise, drawing learnings from past reengineering experiences as guiding principles. The methodology framework should address various aspects, including identifying phases/modules, defining activities, deliverables, prioritizing modules, handling dependencies, establishing ownership, and setting completion dates. A strong project governance, design review, risk management, and a fitting execution-delivery model would enable guardrails for the overall initiative.

2. Show quick wins, gain business trust

Prioritize business use cases and build MVPs to help stakeholders understand what to expect from the new data platform and showcase quick wins. Incorporating changes as per their feedback goes a long way in winning business trust and gaining more support.

3. Proficiency is crucial

Engaging domain and technology experts becomes imperative with processes, technology, and business considerations in place. Experts who have previously dealt with such initiatives and have experience handling these functionalities would bring significant value to the team. They can guide avoiding pitfalls, offer insights on approaches, and assist the team in getting things right the first time.

4. Identify the right Snowflake services and accelerators

Utilize accelerators, frameworks, and methodologies designed for the Snowflake reengineering exercise to avoid pitfalls, expedite the overall implementation, and enable results. Explore new ways to empower your business and consider Snowflake’s capabilities beyond the immediate data processing needs. Use native Snowflake services wherever feasible, from streaming requirements to building applications or AI. Identify the right native Snowflake services to enhance the user experience.

Conclusion

A Snowflake greenfield reengineering program based on the insights and expertise acquired from extensive Snowflake reengineering initiatives is a sure way to avoid hiccups or chances of failure. Such a reengineering program should comprise methodology, framework, jump-start kit, and accelerators, all geared toward expediting the greenfield reengineering program.

Ensuring Snowflake greenfield reengineering initiatives are first-time-right, delivered on time, and cost-optimized requires:

• Consulting with expertise: Consulting should encompass various aspects, offering guidance on business, technology, and processes, such as:
  • Assisting stakeholders in finalizing use cases and defining the scope
  • Recommending a Snowflake ecosystem and tools related to DataOps, ETL, AI, and more
  • Emphasizing data governance, automation, and observability
  • Guiding organizations on change management, strategy, thought leadership, and planning
• Redesign: It involves core data engineering activities, from building the Snowflake platform from scratch to operationalizing it. This phase encompasses tasks such as defining the architecture, creating the data model, building transformation pipelines, conducting testing, ensuring governance, and implementing an automated DataOps platform

LTIMindtree’s Snowflake Reengineering Program offers a proven methodology, framework, jump-start kit, and accelerators, backed by extensive expertise. The program is aimed at expediting the greenfield reengineering to build the Snowflake Data Platform right the first time.