Successful AI Projects Start Here

Inside Forecasting: What Really Sets Strong Forecasting Applications Apart

24.03.2026

Kilian Schneider

Senior Data Scientist

Why Time Series Forecasting Is Rarely a Purely Theoretical Problem in Practice

Time-series forecasting is often viewed as a clearly defined machine learning task: data is collected and preprocessed, models are trained, forecasts are generated, and the results are evaluated using metrics.

In forecasting projects, however, it becomes apparent that forecasts are rarely viewed in isolation as model outputs. In many organizations, they serve as the basis for business-critical decisions and are therefore automatically subject to intensive scrutiny and discussion.

A key area of tension often arises from two factors:

Forecasts are needed across different countries and business units, each of which has its own requirements and data sets.

New forecasting solutions are compared with existing tools, often using the same evaluation metrics.

Technical model quality is therefore only part of the bigger picture. What matters most is whether forecasts can be accepted and put to use within the organization.

The following insights are based on project experience gained from the development and implementation of forecasting solutions in regulated corporate environments at HMS Analytical Software GmbH.

Management Summary: What Really Sets Strong Forecasting Applications Apart

Successful forecasting applications are not characterized solely by high accuracy, for example, but by features that enable their use in an operational environment.

Three aspects stand out in particular from our project experience:

Acceptance stems from transparency, especially when forecasts deviate from previous plans.
Local data sources are business-relevant and must be integrable into a generalized modeling framework.
Example: In the context of the project, it became clear that countries and business units often have additional data pools that are not included in global datasets. Typical examples of local data sources include regional price promotions, country-specific sales structures, local marketing campaigns, or manually maintained sales assumptions that are not reflected in global reporting.
Forecast quality must not be determined by a single metric but must be validated and checked for plausibility.

In addition, a fundamental decision must be made: Should a forecast be generated using an independent and general approach, or should it be tailored to specific business requirements? The answer influences the acceptance and governance of the entire solution.

In HMS projects, it has become apparent that technical forecast quality alone is rarely sufficient if organizational integration and comparability with existing processes have not been clarified.

Technical Deep Dive: How Forecasting Projects Are Implemented in Enterprise Environments

What platforms and technologies are used in forecasting projects? The projects examined were implemented on various platforms. Among others, the following were used:

Platforms

Palantir Foundry
AWS
Snowflake
Databrcks (including MLflow)

Infrastructure and Orchestration

CloudFormation
Lambda
Glue Jobs
Step Functions Bash scripts

Implementation and Modeling

Python
PySpark
Jupyter
Optuna

Forecasting Methods

ARIMA
Prophet
Random Forest
XGBoost
LightGBM
CatBoost

The combination of these technologies reflects typical HMS project requirements: Forecasting must not only be modeled but also integrated into existing data platforms as a reproducible end-to-end process.

These methods demonstrate that, in practice, forecasting often involves both classical time-series methods and machine-learning models—depending on the data available, the desired outcome, and scaling requirements.

Why Explainable AI Is a Key Factor in the Adoption of Time-Series Forecasting

A key lesson learned from the project context is:

Understanding leads to acceptance.

Forecasting systems are not automatically accepted simply because they appear mathematically or statistically sound. Especially when a new application is compared to existing processes or tools, there is a need for explanation.

In the context of the project, it was therefore recommended to provide “hands-on” solutions early on, for example through:

explanatory components (XAI)

visualizations

simulation options for interaction and influence

Explainable AI for time-series forecasting is thus not viewed as an add-on, but as a prerequisite for integrating forecasts into the business’s decision-making logic.

From HMS’s perspective, explainability is therefore not an optional feature, but a central element of product acceptance in forecasting systems.

Explaining Forecast Differences: Why Deviations Are Not Necessarily Model Errors

A common practical approach is to compare new forecasting solutions with existing tools, often using identical evaluation metrics.

This comparison is typically made from a business perspective: The success of a new tool is assessed based on whether it is at least as good as—or better than—the previous method when measured against the existing metrics.

At the same time, it was clearly stated:

Never rely on a single evaluation metric.

From a data science perspective, a single metric is not sufficient to comprehensively assess forecast quality. Forecasts should also be validated against actual values—for example, through random sampling using plots—and against a larger number of metrics.

Deviations between a new forecasting solution and an existing comparison tool are not necessarily an indication of poor modeling. Similarly, differences between forecasts and actual values can have various causes that go beyond a single evaluation metric. The projects revealed that such deviations often stem from structural differences between the tools, such as:

different modeling approaches and mechanisms

different data sources (global vs. local)

differences in data preprocessing

Explaining forecast differences therefore requires clearly specifying in each case what comparison is being made: against an existing tool, against actual values, or across different data contexts.

Forecast Revisions

Why Forecasting Is Hardly Productive Without Feedback

Another lesson learned from the projects was that forecasting systems only work in the long term if feedback processes are built in.

Successful applications thrive on feedback from users and the business. At the same time, it should be defined early on:

when feedback is provided

how feedback is incorporated

how the use of and responsibility for a central forecasting or analytics function is distributed among the local countries or business units

Experience has also shown that business adoption often only takes hold once the business itself assumes responsibility for the forecasting solution—for example, financially or organizationally.

Forecast revisions are thus less of a special case and more of an expected consequence of real-world usage.

Business Story vs. Independent Forecast: The Key Governance Issue

One of the most important technical decisions concerns the focus of the forecast:

Should the forecasting system provide an independent forecast, or should it take business expectations into account?

In the context of the project, this decision was explicitly framed as a trade-off:

Incorporation of business stories or business expectations
Independent forecast results
or a mix of both approaches

Resolving this question was considered essential for acceptance. Without this decision, it remains unclear by what criteria a forecast is “correct.”

This decision has proven to be a pivotal choice in HMS projects, both for the model logic and for subsequent use by business units.

Why Local Data Sources Are Often Crucial for Forecasting Projects

Forecasting projects often start with global data sources, such as sales figures. At the same time, countries and business units often have additional local data sources.

In the context of this project, it was clearly stated that the business often expects these local data sources to be incorporated into forecasts—for example, as indicators.

A forecasting solution should therefore be able to integrate a broad spectrum of local data sources into a generalized modeling framework.

Systemic Context: Operation, Scaling, and Stability Based on Project Experience

Why a Vague Target in Forecasting Leads to False Expectations

Forecasting projects can only be implemented effectively if a target vision has been defined. In one project context, it was explicitly noted that the desired target vision was initially unclear, both in terms of automation and standardization and with regard to the underlying data.

In forecasting, defining the target involves not only technical requirements but, above all, the question of what constitutes a “good” forecast:

Is the primary goal to optimize a specific metric?

Should the existing tool be replaced or supplemented?

Or is transparency regarding forecasting behavior the main focus?

Project experience has shown that, in particular, the internal client’s expectations must be clarified before modeling can be meaningfully evaluated.

Why Data Access in Forecasting Is More Than Just a Technical Detail

A recurring obstacle throughout the project was uncertainty regarding the data source, as well as delays caused by access and authorization issues.

From the customer’s perspective, one expectation stands out above all others:

The relevant data should be complete, up-to-date, and reliably available—especially when forecasts serve as the basis for decision-making.

The project experience can be summed up in one sentence:

No data, no results.

For implementation, this means two things:

From the customer’s perspective: ensuring the availability, accessibility, and quality of the data.

From an implementation perspective: Identifying data issues early and transparently addressing delays.

Forecasting systems are therefore directly dependent on clearly defined organizational data access—not just on model quality.

Why Presenting Results Is Sometimes More Important Than Infrastructure

In a proof-of-concept context, the following recommendation was made: focus more on plots and the presentation of results, and less on infrastructure.

This observation is relevant from a technical standpoint because it illustrates that forecasting projects are not evaluated solely on the basis of technical elegance, but rather on whether the results are comprehensible and assessable to stakeholders.

Conclusion

Effective forecasting applications are not created solely by choosing a single algorithm, but by the ability to make forecasts transparent, comparable, and compatible with organizational processes.

Project experience shows that acceptance stems from understanding, that forecast quality cannot be evaluated based on a single metric, and that local data sources are often crucial to the actual usability of forecasting systems.

HMS therefore positions forecasting not merely as a modeling issue, but as an interplay of data integration, modeling, validation, and business acceptance.