Use Case | Accelerating Transaction Management with Intelligent Document Filing
Technologies Used - AI/ML, Computer Vision, NLP (Natural Language Processing)
Background
Home buying and selling involves significant money, legal requirements, and multiple stakeholders requiring extensive documentation. File Processors are essential in collecting all necessary documents for successful transactions.
However, manually pairing documents to transactions is cumbersome and error-prone, leading to missing or incorrect files, communication issues, and delays in closings, underwriting, and agent commission processing.
​
View the key pain points revealed in the user research that I conducted.
My Role
Lead Product Designer
-
Spearheading comprehensive discovery of the Anywhere Real Estate Transaction Management ecosystem to enhance collaboration between Agents, Agent teams, and Brokerages.
-
Transforming the legacy Homebase platform into an intuitive system for transaction management, Agent commission tracking, document organization, and file processing.
-
Specialized focus on brokerage operations for Coldwell Banker & Sotheby's International Realty.
​​
-
Set up Design Principles to drive adoption and agent, broker & file reviewers & processors.
​
-
Set up the design execution for backlog and grooming user stories within the pod.
Business Goal
Boost File Processing productivity and streamline real estate document management through AI/ML, Computer Vision, and NLP to,
-
Connect transaction data points
-
Enhance document filing accuracy
-
Reduce manual steps for File Processors
Collaborated with Product, Engineering, and Go-to-Market teams to ensure technical feasibility and effective implementation.
Problem Statement
File Processors require a streamlined method to accurately and efficiently assign incoming documents to their respective transactions. This process must enable File Reviewers (licensed individuals) to approve and prepare documents for transaction closing with minimal friction - reducing the time to close a transaction for home ownership.
Solution
The recommended solution involves implementing a unique identifier for each document, ensuring all submissions are linked to the appropriate transaction. This approach will reduce errors, improve tracking, and enhance workflow efficiency.
Hypothesis
AI-powered automatic document assignment would eliminate manual transaction searching while preserving File Processors' established workflows. This Human-AI collaboration was projected to deliver a 10X efficiency gain in document filing operations.
Research Insight and rationale
An extra hour in a File Processor’s day enables roughly 40 more document submissions — a key performance metric.
​
For CB/SIR, this adds up to over 650,000 additional submissions annually, significantly improving file review efficiency.
Result
The project deployment occurred in three strategic phases with immediate impact. Within the first week, our solution automatically processed 16,000 documents directly into "Ready for Assignment" status, eliminating manual compare-and-search operations.
During the August launch week, the system processed approximately 75,000 documents from My Deals, contributing to a monthly total of 324,000 documents—representing a 25% increase in overall document processing volume.
The workflow achieved a 7X increase in document filing efficiency.
This program was done across 3 phases. ​
Current Flow - User flow - Phase 1 | To Provision for a Unique Identifier
Future State - User flow - Phase 2 |Bridge the gap in Documents without a unique identifier.
Document Flow Automation Concept
This concept makes it faster and easier for File Processors to handle documents, even those sent outside company apps.
Using OCR technology and human-centered design, it speeds up submissions and highlights accuracy, so processors can quickly catch errors and move work forward.
BEFORE | 1st design iteration without a unique identifier
• Manual search for a Transaction based on the info in the 'Subject' line (as shown in image below)
​• Accuracy is dependent on skill level of the File processor
• Daily submission average was 300+ Docs/Processor
AFTER | 2nd design iteration with a unique transaction identifier
​Design enhancement to link documents to the relevant Transactions
• System auto-links the document to a Transaction and auto assigns.
• The assigned document is assigned to the address in the 'Subject' line.
This is to align with the natural habits the File processors are trained in.
• File Processors assess the OCR Accuracy of a document or 'return to Agent' for correction. (A Human-in-the-loop concept)
Future State - User flow - Phase 3 | Human-in-the-loop - RLHF
(Reinforcement Learning from Human Feedback)
Human feedback fine-tuned the LLMs into safe, secure, and more accurate rate of confidence in text extraction. Having the human in the loop will also address the hallucinations.
User Flow - PROPOSED | Phase 3 - Automation at Scale & Speed with AI/ML, Computer VisionComputer vision would enhance Optical Character Recognition (OCR) by improving text detection in challenging conditions like poor lighting, varied fonts, or distorted text — enabling faster, more accurate text extraction.
Value Proposition
The proposed AI-driven solution enhances File Processor productivity by automating document identification, assignment, and submission — significantly reducing manual effort and improving accuracy and brokerage operations.
By integrating AI/ML models with Computer Vision and NLP capabilities, the solution delivers the following key benefits:
For File Processors (Primary Users)
• Increased Efficiency - Faster document processing with minimal manual effort.
• Fewer Errors - Improved accuracy reduces misfiled or missing documents.
• Focus on Critical Tasks - Automation frees time for higher-value work.
For Real Estate Brokerages
• Faster Closings - Improved document accuracy reduces delays.
• Better Agent-Broker Relations - Fewer disputes due to lost documents.
• Scalable Solution - Handles more transactions without added staff.
For Real Estate Brokerages (Business Impact)
• Faster Transaction Closures - Improved document accuracy reduces delays in underwriting and closings.
• Stronger Agent-Broker Relationships - Fewer lost or misplaced documents mitigate disputes and improve trust.
The AI-application Strategy
​​1. Image Capture: Extract text from uploaded documents.
2. Text Detection: Computer vision models (e.g., CNNs) identify and categorize text regions.
3. Text Recognition: An OCR engine extracts characters and words.
4. User Interaction: UX will design prompts for users to review OCR confidence and address errors or hallucinations.
5. Model Improvement: Repeated usage will refine the LLM model for greater accuracy.
Document Identification via Unique Identifier
Implementing a unique identifier for each document will ensure that every document is traceable and linkable to a specific transaction — an essential foundation for the proposed AI-driven automation (Phase 1).
AI/ML for Automated Document Assignment
Leveraging Computer Vision and NLP (Natural Language Processing) techniques can effectively extract key data points such as property addresses, transaction IDs, or agent names.
Fine-tuning an LLM specifically trained on real estate terminology, contracts, and document types can improve accuracy in identifying and assigning documents.
Project Scope & Timeline
The project was executed as a 12-month digital transformation program divided into three key phases.
​
Phase 1 - Wireframes and Workflow Improvements
-
Conducted 4 weeks of user research to uncover pain points.
-
Designed user flows to address identified issues.
-
Developed and tested wireframes that connected data points to improve workflow efficiency.
Phase 2 - Tech Feasibility and Product Viability
-
Created user flows and low-fidelity wireframes to assess feasibility and cost.
-
Analyzed the time, effort, and cost implications in collaboration with PM and Engineering.
-
Presented findings to Product and Business leadership for buy-in.
Phase 3 - AI/ML Integration for Enhanced Productivity
-
Proposed a solution to extract and categorize data from documents.
-
Designed a model to train Large Language Models (LLMs) for document management tasks.
-
Integrated AI/ML to improve document processing speed and scale for the brokerage.
​
Timeline
-
Concept Development & Cost Analysis | 8 weeks.
-
User Research | 4 weeks.
-
User Flow Design | 2 weeks.
-
Full Implementation & Testing | Ongoing throughout the 12-month period.
​
This strategic approach ensured improved accuracy, streamlined processes, and increased efficiency for File Processors while reducing delays in closings, underwriting, and agent commissions.
Reframing the UX Double Diamond Framework
For agile delivery of AI-driven user stories
The traditional UX framework, rooted in design thinking, primarily focused on managing a feature backlog. To inspire product and engineering teams to deliver features over a two-week horizon and behind feature flags, I revamped the agile UX delivery model to better balance business objectives with stakeholder expectations.
Success Metric
We would know that the proposed design is successful when;
1. 100% of documents are submitted from My Deals, Agent facing app.
2. There is an increase in the performance metrics of the role File Processors.
KPIs to measure are -
• Number of documents assigned to a file,
• Number of documents submitted for review.
3. Increase in the Design recommendation ROI - Time on Task
User Research and Insights
​
Discovery Phase - User Interviews
A 4-week discovery phase was conducted to understand the challenges faced by File Processors in document assigning and submission. The research combined qualitative and quantitative methods to gather insights.
​
Qualitative Analysis focused on
-
Obstacles faced by participants.
-
Workarounds adopted to address inefficiencies.
-
Time differences between new and experienced employees.
-
Variations in processing time for large documents versus single-page submissions.
​
Quantitative Analysis measured
-
Time on task.
-
Redundant steps in the process.
​
Key Pain Points Identified
-
Some emailed documents lacked address details and had to be manually matched to transactions within 45 days to avoid deletion.
-
Missing transaction IDs prevented documents from being returned to vendors.
-
Video recordings highlighted strained relationships between agents and brokers due to lost documents.
Design Recomendation ROI - Time on Task
CB Brokerage File Processing Data (by month)
Average Submission uploaded - 286 Submissions/Day (in July)
Processing Hours - 6.5 Hours/Shift
[8 hours shift - 1 hour for breaks & lunch-time for meetings, quality checks, and others.]
Per hour average submission uploaded - 44 Submissions/Hour
CB/SIR (Brokerage only) - Total Processors - 62 (at the time)
Measuring current state - Time on task to manually pair a document with a transaction
File Processors varied in skill levels ranging from newly trained (for one US State) to experienced (for cross-US States) based on their understanding of multiple state/county regulations.
Task - 12 participants (with varying years of file processing experience)
Search for the specific transaction and then the folder in which to add the document.
Average Time on Task/Submission,
• Low average - 0:20 secs approx.
• High average - 0:40 secs approx.
• Average - 0:30 secs approx. (note: Average time on task was used for all calculations)
What would it take to do this at scale and be repeatable?
Lo-Fidelity Wireframe - PROPOSED Design concept
Phase 2 - Bridge the gap in Documents without a unique identifier.
To gauge the cost, time, and effort for developing the feature, I created the first design draft.
1. The proposed design would have a drag & drop feature to upload doc files in the transaction details page.
2. Uploaded files will all be passed to the Documents API and receive a unique identifier linked to a transaction.
3. Now, the documents will flow into the already setup Phase 1 User flow.
Time, Effort, Cost Analysis - in collaboration with PM & Engineering teams
Time
3-4 sprints (2weeks cycle) = 8 weeks approx.*
Effort
# of user stories (US) - 8 to 12*
Average size of User Story - 5 points
Conservative Estimation - 60 points* [from 12 (US) X 5 points = 60 points]
Cost
Average cost per story point in TM - $658 (in a month)
Conservative Estimation on cost of project - $39,480* [$658 X 60 pts = $39,480]
* (All calculations are conservative estimates based on 1st draft of wireframes in Phase 2. Wireframes have been made to assess potential time, effort, cost it would take to develop the design to release in production environment start receiving.)
Recap​
​
Human & Al partnership = File Processor is assisted by AI and so the human in the loop is always in control of the validity of the output.
1. At the time when I proposed the Phase 3 concept, Anywhere Real Estate Inc. submitted about 3,000 Docs/Day from MyDeals.
​
2. If we would consider 86% accuracy as a good starting point benchmark for the LLM models.
​
3. This implies that 2,580 (out of 3000) documents can be potentially auto-assigned.
​
4. 14% of the documents will likely have errors or hallucinations; that calculates to 420 Docs, an approximate amount of work that a file processor does in a day.
As such, if the File Processor maintains their usual workflow while addressing hallucinations, the Human and AI partnership could potentially achieve a 10X increase in efficiency.
Dependencies
• Mindset change – Agents send about 65% of documents through email.
• How might we get all Agents to come through MyDeals to submit documents?
• Standardization of documents – classifying documents by categories is essential to support the learning of the model.
• Inconsistent data naming convention between Agent, Customer & Brokerage apps that can affect endpoint mismatch across Applications.
Next steps
Further AI-powered features for Broker & Agent support;
• Design and assess File Reviewers KPIs
• Gain greater insight on benefits the ‘Submit’ tab lends itself towards efficiencies and cost savings to ready transaction files for closings.
• Benefits towards risk mitigation – documents for efficient legal support to Agents.
Further AI-powered features for User Interaction;
• Error detection and fixing them, such as mis-spellings, incorrect dollar value input.
• Detecting fraudulent signatures.
• System generated 'return to Agent' for the documents with error.
Cartus Online