DSS Workflow Fixes

## Summary The meeting focused on addressing critical issues within the Data Services System (DSS) and its interaction with the legacy system (DDIS), with the primary goal of streamlining workflows and reducing manual intervention. Discussions revolved around specific error types, output failures, and proposed solutions to enhance system efficiency and data integrity. ### DSS Audit Queue and Error Handling The DSS audit queue is designed to flag bills requiring operator intervention, but several improvements were identified to make it more effective. The system currently halts bills with validation errors, such as invalid units of measure or service item descriptions, but the process could be optimized by redirecting certain errors automatically. - **Redirecting unit of measure errors to Account Setup:** This would leverage existing vendor and account-level notes stored in DDIS, allowing for faster resolution without manual operator searches in DSS. - **Addressing invalid observation types:** A recurring issue was identified where DSS sometimes creates its own, invalid observation types (e.g., "non fuel charge charge") instead of using the correct, system-recognized ones (e.g., "charge PR"). These errors are not always visible to the team until the bill fails later in the process. - **Enhancing communication with notes:** When bills are rerouted from DSS to other workflows like Account Setup, it was requested that a note be automatically appended explaining the reason for the rerouting. This would prevent confusion and save time for operators who currently have to investigate the cause manually. - **Rerouting logic for complex bills:** The logic for handling bills over six pages was discussed. Currently, these should be kicked out of DSS, but they are sometimes automatically picked up again, creating a loop. A proposal was made to send any bill with an existing client, vendor, and account number directly to the data audit queue instead. ### Output Service and Data Capture Issues Significant problems were reported with the output service, where data captured in DSS leads to failures when sent to the downstream UBM system. These issues often result in entire batches being rejected, necessitating manual reprocessing of all bills in the batch. - **Broken batches from blank meter lines:** A major cause of batch failures occurs with deregulated accounts and zero-usage bills. The system captures service dates on the supply meter but no usage or charges, resulting in a blank line on the output file that the UBM system cannot ingest. - **Re-emergence of a previously fixed issue:** This problem with supply meters had been resolved but has recently resurfaced, leading to a high volume of failed batches, including for new clients, indicating a regression in the system. - **Proposed solution of splitting batches:** To mitigate the impact, it was suggested that batches be broken into individual bills for output. This would ensure that only the problematic bill fails, rather than causing the entire batch to be rejected. ### Observation Type and Data Integrity Problems Beyond the output service, critical data integrity issues were highlighted where observation types and units of measure are being incorrectly altered during processing, leading to bills getting stuck. - **Blank or incorrect observation types:** Bills are being captured and sent to "ready to send" with blank observation types or observation types that do not exist in the system, forcing teams to manually pull them back and investigate. - **Incorrect unit of measure assignment:** A specific issue was described where the unit of measure on a default meter is changed to dollar signs, creating a cost line with no usage data. This also causes bills to fail downstream. - **Manual workaround and its limitations:** The current fix for these data issues involves manually using the "clear data" button, which reverts the units of measure and allows for reprocessing. However, this is a time-consuming and reactive solution that highlights a need for a more robust fix within DSS. ### Proposed System Improvements and Next Steps The discussion concluded with a focus on short-term fixes and longer-term system enhancements to address the root causes of these recurring problems. - **Short-term data routing with context:** An immediate action item is to configure DSS to push bills with specific errors, like unit of measure issues, to DDIS (Account Setup). Crucially, this will include sending detailed notes that identify the problematic meter by commodity type and meter number, along with the specific error message. - **Targeting key pain points:** The team committed to investigating several high-priority issues, including blank observation types, the "do not use" meter list, and the root cause of the broken batches from blank supply meter lines. - **Long-term vision for DSS:** The ultimate goal is to fully defang the legacy output service by having DSS handle output natively, thereby eliminating reliance on the legacy system and its associated failure points. Ensuring the DSS pre-audit queue is effective is a critical step toward this goal.