Data collection processing#

Data Collection is the write-back layer that takes collected contribution input, an uploaded input workbook or one received in a monitored mailbox, and writes its values back to the target system, for example a TM1 or Planning Analytics cube or a SQL table. This guide is for the administrator who runs and troubleshoots that processing.

Data Collection is part of the licensed Contribution capability. Without it, the Data Collection settings show "Data Collection is not included in your license," and you contact Reportworq to enable it.

When to use it. After contribution forms are submitted or approved and a campaign export runs, or whenever input workbooks arrive by upload or into the monitored mailbox and need to post to the source planning system.

Before you start#

• You need the Contribution capability licensed (it includes Data Collection) and Administrator rights.
• A write-back target must be reachable: TM1 / Planning Analytics, SQL, Workday Adaptive, or Anaplan. Vena is a reporting source only and cannot be a write-back target.
• For the mailbox intake path, have a dedicated mailbox and its credentials ready. See the section below, Set up the monitored mailbox.
• Data Collection processes input workbooks. It is not the destination of a contribution campaign's Export action, which writes to the target system itself. See Two write-back paths below.

Two write-back paths#

Reportworq writes contribution data back to a planning system in two ways. They share the licensed Contribution capability and nothing else: they are separate pipelines, they reach the target by different means, and a given round uses one or the other.

• Write-back through Data Collection (this guide). Input workbooks arrive by upload or in a monitored mailbox, and the Data Collection engine processes them. The values are posted by the hidden write-back formulas inside the workbook (DBS, DBSW, DBSS, RWSQLUPDATE, and the equivalents for other targets), which the engine evaluates in write-back mode.
• Write-back through a contribution campaign. Contributors enter values in a generated form, and the campaign holds them in its own runtime. Nothing reaches the planning system until the campaign's Export action runs, on a trigger, on a schedule, or by hand. The export builds a table of the collected values and hands it to an export provider, which writes to the target directly. For that path, see Author a contribution campaign.

A campaign export does not go through the Data Collection engine. It does not queue a write-back item, and it does not evaluate the hidden write-back formulas. If you are looking for a campaign's values in the write-back activity list, they will not be there; a campaign's exports are recorded in the campaign's own activity.

The rest of this guide covers the Data Collection engine path.

How write-back items are processed#

Write-back items enter one of two ways and then move through the same processing pipeline.

1. Upload. An administrator or author uploads an input workbook from the write-back activity view (Upload…). With Automatically process uploaded files turned on, the upload processes on arrival. Otherwise, select the pending item and run Process.
2. Mailbox. The monitored inbox is polled on a configured frequency, and matching messages become items. Select Check Inbox to poll on demand.

Each item begins Pending, moves to Running as the engine writes its data to the target, and finishes Complete. The write-back activity grid lists every item with its Filename, Submitter, Uploaded, Processed, and Status, plus per-row Run and Details actions.

After a file is processed, an audit record of the write-back outcome is written to the input archive.

Configure Data Collection#

Open Configuration ▸ Data Collection to set the processing behavior.

Let consolidated-cell writes through (Proportional Spreading)#

By default, Reportworq rejects any attempted update to a Planning Analytics consolidated cell, matching the Planning Analytics Excel add-in. If a form appears to "lose" a top-level input, that input was most likely rejected for this reason.

To allow a submission to a consolidated cell, turn on Proportional Writeback. The value is then sent to Planning Analytics with the proportional-spread marker (a P prefix), so Planning Analytics allocates it across the leaf cells based on their existing data.

Set up the monitored mailbox#

Open Configuration ▸ Data Collection ▸ Mailbox Settings and complete the connection.

If Data Collection is enabled with Provider set to Microsoft 365 and no credential is selected, saving shows an error asking you to pick one, or add one under Settings > Integrations > Shared Credentials.

Do not validate the mail server's TLS certificate: when to use it, and the trade-off. Reportworq 5 accepted a mail server's certificate silently; Reportworq 6 validates it strictly by default, so a mailbox that collected fine on v5 can fail to connect after a repository import. Turn this on only for an internal mail server you trust, since the connection is still encrypted but the server's identity is no longer verified. Each poll that accepts a bad certificate is recorded as one warning in the application log naming the host. If Reportworq cannot connect at all, the application log error now names the protocol, server, and port it tried, for example Could not connect to IMAP server mail.example.com:993 (SslOnConnect): ....

Exchange Online mailbox: two traps#

If the monitored mailbox never picks up messages against Exchange Online, it is almost always one of these two causes:

• POP3 is disabled on the account. Reportworq polls the monitored mailbox over POP3. Exchange Online supports POP3 but it is disabled by default and must be explicitly enabled on the specific mailbox account, per account, in the Exchange admin center.
• Basic Auth is deprecated. Microsoft has retired Basic Auth for POP3 and IMAP in Exchange Online, so the Basic Authentication mailbox option fails against it. Set Provider to Microsoft 365 instead, and choose the shared Microsoft 365 credential to authenticate with. See Microsoft 365 OAuth setup.

Prune the monitored mailbox#

Reportworq records processed mailbox messages in its own internal database so it never reprocesses the same message. That means the mailbox itself is not emptied by processing, and processed messages accumulate in it over time.

Apply a retention rule to the monitored inbox, for example archive or delete messages after a set period, so the mailbox does not grow unbounded. The processed-message history that Reportworq keeps is separately bounded by the Days to Keep setting.

Notes and limits#

• Data Collection is unavailable without the licensed Contribution capability.
• Consolidated-cell writes are silently rejected unless Proportional Writeback is on. This is the first thing to check when a top-level submission does not land.
• Write-back targets are TM1 / Planning Analytics, SQL, Workday Adaptive, and Anaplan. Vena is a reporting source only and cannot be a write-back target.
• The write-back input formulas that make a form post its data (DBS/DBSW for Planning Analytics, RWSQLUPDATE/RWSQLUPSERT for SQL, AIINPUT/AIMODELEDINPUT for Workday Adaptive, and ANAPLANINPUT for Anaplan) live in hidden rows by design and render as #NAME in the contributor's view. Do not un-hide them.

Going deeper. For live memory pressure on the contribution worker fleet during a large collection round, see Contribution processes. For symptom-to-fix pairs, see Troubleshooting index.