Use Only Information Available at the Time
The model used only historical records that would have been available before the simulated trip. Actual delays and other future information were excluded.
Historical Testing
How Well Did MakeItBy's Estimates Match Historical Results?
MakeItBy estimates the chance that each scheduled flight arrives by your deadline. We tested those estimates across historical flight comparisons, then checked how reliability changed as scheduled arrival times moved closer to the deadline.
96%
A Real Tradeoff in Most Tests
Most Historical Comparisons Involved a Real Tradeoff.
Across 467,652 simulated historical comparison scenarios, about 96% left more than one flight worth considering. In the other 4%, the latest scheduled flight was also at least as reliable as every earlier choice.
5h 18m
average gap between the earliest and latest scheduled arrivals
9.6 percentage points
average spread in estimated deadline reliability across the flights compared
Estimated: 94% · Observed: 95%
estimated versus observed deadline success for the leading choices
These are historical tests, not a forecast for your next flight.
Current weather, air-traffic restrictions, maintenance, gate changes, and other
day-of-travel conditions are not part of these estimates.
Tradeoff Structure
Most Historical Comparisons Had More Than One Competitive Flight.
In most tests, no single flight was both scheduled to arrive later and at least as reliable as every other option. That means the choice depended on how much scheduled arrival convenience the traveler was willing to trade for deadline reliability.
How Often Was There a Real Tradeoff?
Share of simulated historical comparisons
96%
had more than one reasonable option
4%
had one clear latest option
These were simulated historical comparison scenarios, not 467,652 unique travelers. The historical test used every flight in each tested route and arrival window. The live product is narrower: it compares only the flights you enter.
Estimates vs. Outcomes
The Estimates Stayed Close to What Actually Happened.
If the model says a group of comparable flights has about 94% deadline reliability, historical calibration asks whether roughly 94% of those flights actually made the deadline.
Flight group
Estimated
Observed
Earliest scheduled choice
98.1%
98.2%
Tradeoff choices
94.4%
95.0%
Latest scheduled choice
88.6%
89.5%
Historical calibration does not guarantee a future result.
Simple Buffer Comparison
Model-Based Rules Often Identified Later Flights With Similar Historical Deadline Success.
We compared model-based reliability thresholds with simple rules such as “take the latest flight with at least three hours of scheduled buffer.” A 96% reliability target, for example, means choosing the latest flight whose estimated chance of making the deadline was at least 96%. The examples below include only historical scenarios where both rules could choose a flight.
95% reliability target
vs. 2.5-hour buffer
About 35 minutes later
scheduled arrival
About 2.0 additional deadline misses per 1,000 historical comparisons.
96% reliability target
vs. 3-hour buffer
About 39 minutes later
scheduled arrival
About 0.6 additional deadline misses per 1,000 historical comparisons.
97% reliability target
vs. 4-hour buffer
About 59 minutes later
scheduled arrival
About 1.1 additional deadline misses per 1,000 historical comparisons.
These reliability targets are evaluation settings, not MakeItBy recommendation settings. The product does not choose how much reliability you should trade for a later scheduled arrival.
Stability
The Same Pattern Appeared Across All Tested Dates, Lead Times, Deadlines, and Window Lengths.
In the 96% reliability target versus a three-hour buffer, the model-based rule allowed a later scheduled arrival in every test date, booking lead time, arrival deadline, and comparison-window length we checked.
6 of 6
test dates
18.6 to 81.9 minutes later
4 of 4
lead times
38.2 to 38.9 minutes later
3 of 3
deadlines
20.4 to 69.2 minutes later
3 of 3
window lengths
34.4 to 42.5 minutes later
Later Scheduled Arrival Does Not Mean Better in Every Test.
Historical deadline success sometimes favored the simple buffer rule and sometimes the model-based test. The consistent result was a later scheduled arrival with only a small difference in observed deadline success.
Test Method
We Rebuilt Each Historical Flight Set Before Checking the Outcomes.
Rebuild the Tested Flight Set
Every historical flight in the tested route and arrival window was retained before the model-based and simple-buffer comparisons were evaluated.
Compare With What Actually Happened
We checked whether each selected historical flight actually reached its scheduled destination by the simulated deadline, including cancellations and diversions as failures.
33.0 millionhistorical flight operations in the validated dataset
Jan 2022–Jun 2026historical data period
50 states + DCU.S. domestic nonstop scope
Model Development
We Compared Five Model Families Before Choosing the Production Approach.
We compared five model families on forecast accuracy, calibration, stability, and decision behavior. The final approach had the strongest overall forecast accuracy, and no alternative showed a clear enough product-level advantage to replace it.
Model families tested
5
Training examples
6.6 million
Data through
June 2026
Limitations
Historical Reliability Is Useful Evidence, Not a Guarantee.
The Live Comparison Is Limited to the Flights You Enter
MakeItBy does not claim to search every scheduled flight at launch. It resolves and compares the candidate flights you provide.
Schedules Can Change
The historical model does not include the chance that a future published schedule is later retimed, removed, or assigned to a different operation.
Day-of-Travel Conditions Still Matter
Weather, air traffic, maintenance, and other current operational conditions can change what happens after the comparison is made.
24-Hour Comparison Limit:
MakeItBy only scores submitted flights scheduled to arrive from the deadline itself
back through 24 hours before it. Flights outside that window are not part of the
supported launch comparison.
One Important Testing Limitation:
the historical comparison reconstructed final schedules from U.S. Bureau of Transportation
Statistics records. It could not reproduce the exact future schedule a traveler would
have seen months before departure.
For a cruise with a fixed boarding deadline, see how to calculate your airport-arrival deadline before comparing your flights.
For a separate descriptive comparison of historical flight outcomes, see July vs. October Flight Reliability. That analysis is not a test of MakeItBy's predicted probabilities.