A higher-education institution needed to transition its HR and payroll operations from ADP (Automatic Data Processing) to Ellucian Banner HR/Payroll SaaS. That migration also required the institution to move its retirement contribution process with TIAA, its retirement recordkeeper, to the new SaaS architecture. Payroll data still had to reach TIAA using the SPARK Data Format, an industry standard for contribution remittance, and the resulting feedback had to be reconciled with the institution’s payroll records.

The existing process handled the exchange, but staff processed the TIAA feedback manually. Payroll staff downloaded a large feedback file, identified the small subset of fields needed for reconciliation and manually corrected exception records. The migration therefore involved more than replacing ADP as the payroll source. It was also an opportunity to redesign the full contribution and reconciliation cycle for Banner SaaS rather than carry the manual process forward.

The project therefore called for ABCloudz to redesign the integration around Ellucian Data Connect, Ellucian’s data-orchestration layer, and Ellucian Ethos, the API layer used to access Banner SaaS data, with separate outbound and inbound pipelines for contribution remittance and automated feedback reconciliation.

Move TIAA retirement contribution processing to a Banner HR/Payroll SaaS architecture while preserving the required SPARK exchange and automating the feedback reconciliation loop.

From an ADP-based process to a SaaS-ready integration

In the previous process, ADP generated retirement contribution data for TIAA in pipe-delimited SPARK files. TIAA processed the contributions and returned a feedback file containing confirmations, errors and exceptions.

The most labor-intensive part came next. Payroll staff downloaded the feedback file from TIAA’s SFTP site, opened it as a text document and manually searched the full file for the six or seven fields needed for reconciliation. A typical payroll cycle produced fewer than 25 relevant exception records, which staff identified and corrected manually in ADP.

The assessment also found several legacy custom Banner tables associated with earlier employee and benefit processes. They were no longer part of the required retirement workflow, so the modernization did not recreate them in the new architecture.

A detailed TIAA feedback file came back every payroll cycle, but identifying actionable exceptions and reconciling them with payroll data remained a manual process.

Redesigning the retirement contribution flow for Banner SaaS

The Future State makes Banner HR/Payroll SaaS the authoritative source for worker, employment and payroll deduction data. Ellucian Ethos provides API access to that data, while Data Connect orchestrates transformations, secure file exchange and processing state.

The design separates the lifecycle into two pipelines: one builds and sends TIAA remittance files, while the other retrieves TIAA feedback and reconciles it against Banner payroll data.

How do you preserve a file-based TIAA exchange while moving the payroll source to SaaS and removing the manual reconciliation loop?

Building the outbound SPARK remittance pipeline

The outbound pipeline runs on a payroll-aligned schedule. It queries Banner SaaS through Ethos for employees and retirement deductions that fall within the current processing window, then normalizes those records for TIAA.

This step requires more than forwarding fields. Banner deduction and plan information has to be translated into the contribution-source semantics and formatting expected by SPARK. Data Connect then assembles the required Header, contribution detail and Trailer records, writes a pipe-delimited file and transfers it to TIAA over SFTP. Credentials are managed through Data Connect secrets.

Banner payroll data and TIAA contribution semantics do not map one-to-one. The integration therefore needs an explicit transformation layer for contribution sources, plan relationships and SPARK formatting.

See how Banner payroll data is transformed into a TIAA SPARK remittance file

Building the SPARK record structure

The pipeline works with the SPARK structure required for contribution remittance rather than producing a conventional CSV with column headers. Its output consists of:

  1. a Header record;
  2. one or more Detail_CR contribution records;
  3. a Trailer record.

The records are ordered according to the applicable SPARK specification and written as UTF-8 pipe-delimited lines.

Conceptually, the transformation looks like this:

Banner SaaS concept Integration logic SPARK output concept
Worker and person identifiers Select the permitted participant identifier and apply required formatting or masking Participant identification
Payroll deduction Normalize amount or percentage and associate it with the relevant retirement plan Contribution amount
Deduction or plan classification Map institution payroll semantics to the required contribution-source definition Contribution Source Code
Plan and source information Derive the combination needed by the TIAA layout Plan/source fields
Payroll calendar or configured processing period Establish the relevant payroll date context Payroll/effective-date fields
Employer or organizational context Populate required static or derived values Employer-related SPARK fields

Some values can be taken directly from Banner data exposed through Ethos. Others, such as combined plan/source derivations or contribution-source mappings, are calculated in the Data Connect transformation layer.

An optional Data Connect serverless API can centralize those derivations where needed. It can also enforce formatting rules such as zero-padding and apply permitted SSN formatting or masking without introducing a new database dependency.

From normalized records to delivery

The processing sequence is:



Read the current processing boundary

        ↓

Query changed employees and retirement deductions through Ethos

        ↓

Normalize payroll and plan data

        ↓

Apply SPARK mappings and formatting

        ↓

Create Header + Detail_CR records + Trailer

        ↓

Write the pipe-delimited remittance file

        ↓

Upload the file to TIAA over SFTP

        ↓

Record the new successful processing boundary

The pipeline can also run in dry-run mode, generating the SPARK output without transferring it to TIAA. This provides a controlled way to validate an off-cycle or test execution before external delivery.

Automating TIAA feedback reconciliation

The inbound pipeline handles the return path from TIAA. After TIAA produces its feedback file, Data Connect retrieves it from SFTP and parses the complete file rather than extracting only the small subset previously reviewed by staff.

Header, Detail and Trailer records are separated and normalized into structured data. The pipeline then queries the corresponding Banner SaaS employee and payroll information through Ethos and matches the two sides using employee and payroll-period context.

Contribution amounts and plan/source information are compared. TIAA-specific reason codes are retained with the reconciliation data, while mismatches are identified as exceptions. The pipeline stores the normalized results in SaaS-compatible storage and makes operational metrics and exception information available through Ellucian Insights.

Banner SaaS now drives the TIAA remittance process, while an automated inbound flow reconciles TIAA feedback against the institution’s payroll data.

See how the feedback pipeline turns a TIAA file into reconciled records and actionable exceptions

Parsing and normalizing the feedback file

The pipeline retrieves the latest applicable TIAA feedback file and parses all fields defined for the exchange. Records are first classified by their structural type:

  • Header;
  • Detail;
  • Trailer.

Detail records are then normalized into a tabular reconciliation dataset. Structural fields remain part of file processing, while participant, payroll, plan, contribution and reason-code data become available for validation.

The normalized feedback can include information such as payroll date, participant identifiers, employee name, employer plan information, contribution amounts, source or allocation information and TIAA reason codes.

Correlating TIAA feedback with Banner SaaS

For the relevant payroll period, the pipeline retrieves corresponding Banner data through Ethos. Representative relationships include:

TIAA feedback information Banner SaaS comparison context
Payroll date Payroll calendar or processing period
Employee identifier Banner person identifier
Participant name Person data
Plan identifier Retirement plan or deduction context
Contribution amount Payroll deduction amount
Contribution source Deduction plan and contribution classification
Allocation information Deduction percentage or amount plus transformation logic
TIAA reason code Preserved as TIAA-specific reconciliation information

When no Banner equivalent is required, TIAA reason codes remain part of the reconciliation record rather than being mapped, preserving the operational context returned by TIAA.

Comparing the two sides

The core decision flow is:

Feedback reconciliation flow

At a simplified level, the logic can be represented as:



for each normalized feedback detail record:



    identify employee and payroll period

    retrieve corresponding Banner payroll data



    normalize comparable contribution and plan/source values



    if amounts and plan/source context align:

        store as reconciled

    else:

        flag as an exception



    preserve TIAA reason information with the result

The comparison is semantic as well as numeric. Banner may expose a payroll deduction as an amount or percentage associated with a plan, while TIAA represents contributions through its own source and allocation structure. The transformation layer aligns those concepts before the records are compared.

Specialized source distinctions, such as pre-tax, Roth, employer-match or adjusted contribution categories, can therefore require mapping logic rather than a direct field-to-field comparison.

Turning reconciliation into operational information

The pipeline writes both normalized and reconciled results to a SaaS-safe dataset or object-storage target. Ellucian Insights receives operational metrics keyed to the payroll context, so payroll teams can focus on exceptions instead of manually interpreting the complete feedback file.

The same design preserves the full feedback record while presenting the actionable subset separately. That distinction keeps reconciliation traceable without forcing every operational user to work directly with the raw TIAA file.

Making scheduled processing safe to rerun

Both pipelines maintain processing state so recurring payroll runs can continue from the last successful processing point instead of starting with a full extraction each time.

For outbound processing, the integration stores the last successfully processed activity boundary and uses it to select changed or active deduction data on the next run. An explicit starting-date override supports backfills, allowing data from an earlier date or payroll period to be reprocessed. The inbound pipeline separately tracks the last processed feedback context.

The design also supports manual executions. Outbound processing can use dry-run mode, while feedback processing can be triggered again when required. If an expected TIAA feedback file is not yet available, the scheduled process retries with exponential backoff, increasing the delay between successive attempts.

See how incremental state, reruns and backfills are controlled

Outbound processing state

The outbound pipeline uses a persisted activity timestamp as the boundary between runs.

A sanitized representation of that state is:



processing_state = {

    last_activity_date,

    last_generated_file,

    record_count

}

The recurring flow is conceptually:



state = read_processing_state()



from_date =

    activityDateFrom override

    if an explicit backfill boundary was provided

    else state.last_activity_date



records = query Banner data through Ethos

          using activityDate / lastModifiedDate filters

          from from_date



normalize and transform records

generate SPARK file



if processing succeeds:

    state.last_activity_date = maximum observed activity date

    state.last_generated_file = generated file

    state.record_count = processed record count

    save_processing_state(state)

The override separates normal incremental processing from controlled backfill execution without changing the recurring pipeline design.

Feedback processing state

The inbound side tracks progress differently. Rather than relying on a Banner activity timestamp, it records the feedback file and payroll context already processed.

Conceptually:



feedback_state = {

    last_processed_feedback_file,

    payroll_date,

    processing_counts

}

A scheduled run identifies the next applicable feedback file, processes it and updates state after the reconciliation cycle completes.

If the expected file has not arrived yet, the pipeline retries with exponential backoff. An on-demand trigger provides a separate path for intentional reprocessing.

Together, these state mechanisms support controlled reruns without forcing a full reload on every execution.

What changed after the redesign

The modernization changes both the integration architecture and the operational process around it.

Banner HR/Payroll SaaS becomes the payroll source for the TIAA integration. Retirement contribution processing now uses the institution’s new HR and payroll platform as its payroll source rather than relying on the previous ADP-based process.

The exchange operates through supported SaaS integration mechanisms. Ethos provides access to Banner data and Data Connect handles orchestration, transformations, state and secure SFTP exchange without recreating the legacy custom Banner tables.

TIAA feedback becomes a structured reconciliation workflow. The complete feedback file is parsed and compared with Banner payroll data automatically instead of requiring staff to search manually for the relevant exception records.

Exceptions become operational data. Reconciliation results and error information can be stored, analyzed and surfaced through Insights, separating actionable exceptions from the full technical feedback record.

Modernize the business loop, not just the transport. Replacing the payroll source addressed the outbound dependency; automating feedback reconciliation completed the process redesign.

Frequently asked questions

Why was the TIAA integration redesigned during the move to Banner SaaS?

ADP was the payroll source for the existing retirement contribution process. As Banner HR/Payroll SaaS became the authoritative source for worker and deduction data, the TIAA exchange also needed to be rebuilt around that new architecture. The project addressed both outbound contribution remittance and the manual feedback process that followed it.

Does the solution still use TIAA SPARK files?

Yes. TIAA remains the retirement recordkeeper and the solution preserves the required SPARK-based exchange. Data Connect assembles the Header, contribution detail and Trailer records, writes the pipe-delimited remittance file and transfers it to TIAA over SFTP.

How does Ellucian Data Connect access Banner payroll data without direct database access?

Data Connect queries Banner SaaS data through Ellucian Ethos resources. The scheduled pipelines use activity-based filters to retrieve the relevant employee and payroll deduction records and then perform the required transformations inside the integration flow.

Why are there two Data Connect pipelines instead of one?

The two directions have different processing responsibilities and state. The outbound pipeline selects Banner payroll data, transforms it into SPARK and sends it to TIAA. The inbound pipeline retrieves TIAA feedback, parses it, compares it with Banner data and produces reconciliation results and exceptions.

How is TIAA feedback matched back to Banner payroll data?

Feedback records are normalized and correlated with Banner records using employee and payroll-period context. The pipeline then compares contribution amounts and plan/source information, preserves TIAA-specific reason information and flags mismatches as exceptions.

What happens when Banner and TIAA represent contribution data differently?

The integration applies transformation logic before comparing or sending the data. Banner deduction and plan concepts can be mapped to the contribution-source and allocation semantics required by TIAA, allowing the two data models to be reconciled without assuming direct field-for-field equivalence.

Can an earlier payroll period be reprocessed?

Yes. The outbound pipeline supports an explicit activity-date override for backfill processing, and both sides maintain processing state. The feedback pipeline can also be triggered on demand when reprocessing is required.

Does this integration use Ethos Events?

No. This implementation is schedule-based. Data Connect queries the necessary Banner data through Ethos APIs using activity-based filtering rather than consuming Ethos Events, Ellucian’s change-notification mechanism for responding to resource changes.

What role does Ellucian Insights play in the solution?

Insights provides operational visibility into reconciliation results and exceptions. It receives metrics from the integration so payroll teams can focus on rejected or mismatched records instead of reviewing the entire feedback file manually.

Modernizing a similar integration?

Moving an enterprise application to SaaS often affects more than the primary system. File exchanges, custom transformations, reconciliation routines and operational processes around it may need to be redesigned as well.

If you are moving Banner integrations to SaaS or need to replace a legacy payroll, benefits or third-party integration with a supported Ellucian architecture, contact ABCloudz to review your current process and design the right modernization path.

Ready to start the conversation?