Stage 4 - Testing and Validation¶
Testing happens inside Pipeline Generation, not after it. Understanding what is tested - and what is not - matters more than any other single thing on this page.
There is no Stage Page to visit and no button to press. Stage 4 is numbered here because it is a distinct piece of work with results you need to read, but SLIM does it as part of Stage 3, on the way past. See How Modernization Works.
What gets tested¶
Both representations are tested, separately, each in its own terms.
The functional representation gets tests of the individual pieces of logic - the transformations, the business rules, the way errors are handled - and tests that put the whole flow through from end to end. It then gets a demonstration run: the implementation is exercised once, all the way through, with realistic sample data. That last one exists to catch what a set of individual tests can miss, which is a collection of parts that each behave correctly and do not work together.
The SnapLogic side gets test pipelines - pipelines whose only job is to exercise other pipelines. There is one for the shared expression library, checking that each reusable piece of logic returns what it should, and one for the data flow, pushing records through the pipeline and checking what comes out the other end. Where the integration was built as a parent and children, each child gets its own flow test and there is a final test of the parent driving them all in order.
Every test SLIM writes, SLIM also runs, and the results are written into the same folders as the pipelines.
What the tests run against¶
Not your systems. Nothing in this Stage touches them.
Every external connection in a test pipeline is replaced with a stand-in. A database query becomes a small block of fabricated records. A write to a file share, an upload to cloud storage or a call to an API becomes a write to a local file. The volumes are deliberately small - tens of records rather than a production day - because the point is to prove the logic, not the throughput.
The pipelines you will actually deploy are a different matter. Those carry the real connections, and running them would need live accounts against your real systems, which SLIM does not have and is never given. They are checked rather than run: each one must compile cleanly, and whether it did is recorded in the completion report. The accounts and credentials they will eventually need are listed for you in the migration plan, for you to create when you deploy.
So be clear about what a green result is evidence of. It says the logic holds together and produces the shape of output that was intended of it. It says nothing about whether your database is reachable, whether the credentials are right, whether your real data looks like the sample data, or whether the volumes are survivable. Those questions belong to the environment, and they are answered after deployment.
Reading the test results¶
The results are in the completion report in each of the two folders. The one under /python carries the test run output and how much of the implementation the tests covered. The one under /snaplogic carries a section for each kind of test giving its status, how many tests there were, how many passed, how many failed and how many attempts it took, followed by a table of any failures that were still outstanding.
Four things to look at, in this order.
Ignore the heading. Both reports are headed Migration Status: COMPLETE whatever actually happened, because that is what the template says. The heading tells you the document was written, not that the work succeeded. The status line and the numbers underneath it are the result.
A total of zero tests is not a pass. The check SLIM applies is that no test failed, and no test failing is trivially true when no test ran. A section reporting Status: PASSED against a total of zero is reporting nothing at all, and everything downstream of it is unevidenced. If you see it, treat that integration as untested and investigate why.
Read the outstanding failures table if there is one. Its presence means tests were still failing when SLIM stopped working on them. This is not concealed, but it sits below the summary rather than in it, and a reader skimming the top of the document will miss it.
Look at what was changed to make a test pass. One of the repairs available to SLIM is to correct what a test expects rather than the thing the test is checking, and sometimes that is the right repair - a test can genuinely be wrong about an expected value. But it means a green result is not by itself proof that the new behavior matches the old integration. The section recording what was carried back from testing, and the comparison document, are where you see which kind of fix was applied.
Where you attached a design document there is also a conformance report, which is a separate check of the pipelines against that design rather than against themselves. Supporting Documents covers it, including the condition under which it is produced.
Fix and retry¶
Testing here is a loop rather than a step. A test fails, SLIM works out why, changes something, and runs it again.
The loop is bounded. There are a few attempts on the functional representation and fewer on each SnapLogic test pipeline, and the limits are deliberate: a run that kept trying indefinitely on a stubborn failure would spend your money without converging on anything.
When the limit is reached SLIM stops and writes down what is still broken - which tests, what the errors were, and the likely cause where it worked one out. It does not quietly drop them, and it does not present them as passes. The consequence is worth stating plainly: an integration can finish with failing tests recorded against it and still be reported as finished. That is why the section above matters more than the pill on the Stage Page.
What is repaired is not confined to the tests. Where a test exposes a fault in a pipeline or in the shared expression library, the correction is applied to what you are going to deploy - see Stage 3 - Pipeline Generation.
What is still yours to do¶
SLIM’s testing establishes that the generated logic is internally consistent and behaves the way the plan said it should. It cannot establish that it does what your business needs, and it has not been near a real system.
That leaves a sequence which is yours, in this order.
Read the comparison documents against what you know. Behavior that was deliberately not carried across is listed, and so is anything SLIM could not resolve. This is the cheapest review you will do; see Verifying the Output.
Create the connections in SnapLogic. The migration plan lists what each pipeline needs. Credentials are yours to supply, and SLIM neither has them nor asks for them.
Run in a development or sandbox environment, against real connections and representative data, and compare the result against what the original integration produces. This is the first point at which anybody finds out whether the modernization works; see Stage 5 - Platform Deploy.
Then production, through whatever change process you already have for it.
Warning
Never point the generated test pipelines at a production environment. They were built to exercise logic with fabricated data, and their stand-in writes go wherever they are told. Given real connections in a real environment, a test pipeline is capable of writing where nothing should be written. Keep them in a sandbox, or leave them out of the deployment altogether - Stage 5 - Platform Deploy covers choosing what goes.