Announcing the Autonomous Cleaning and Maintenance Management Benchmark – Now Open Source

We are excited to announce the release of our Autonomous Cleaning and Maintenance Management Benchmark as an open source project. This benchmark provides a comprehensive checklist of key capabilities required for systems managing cleaning and maintenance operations autonomously.

The site is available at: https://tidyapp.github.io/maintenance-management-automation-benchmark/
The benchmark is open source on github: https://github.com/TIDYAPP/maintenance-management-automation-benchmark

Why An AI Benchmark for Cleaning and Maintenance Management?

We at TIDY feel like we are close to fully automating cleaning and maintenance management, so no human is required. We expect to achieve roughly human level cleaning and maintenance management by the end of 2025, and superhuman by the end of 2026. Our approach of having humans in the loop allow us to serve customers as if AI is ready, and deeply understand the remaining issues. As we get close to and surpass human level, we expect to see more edge cases come up and we plan to add the missing pieces to this benchmark. We could be wrong on the dates, but after 10 years in the business we seem close.

We are throwing down a challenge to the industry. We are happy to compete and figure out how to create completely autonomous cleaning and maintenance operations.

What Is This AI Benchmark for Property Management?

The benchmark outlines the critical features that a fully autonomous cleaning and maintenance management system should have. While actual cleaning and maintenance tasks may still require human work, the focus here is on automating complex coordination—managing vendors, scheduling, quality control, compliance, and overall business operations.

To be effective, a system must handle both routine operations and unusual or emergency situations. The checklist details standard features and challenging edge cases, such as:

• Basic scheduling and vendor coordination.

• Handling emergency situations.

• Managing multi-party disputes without human intervention.

The benchmark is divided into eight main categories:

1. Property Understanding: Create a digital twin by mapping property data, assets, surfaces, and more.

2. Task Requirements: Define and manage work needs based on property details and client preferences.

3. Vendor Management: Coordinate service providers with cost and timing optimization.

4. Schedule Management: Handle job scheduling and manage unexpected edge cases.

5. Quality Control: Verify that work is completed to the required standards.

6. Property Updates: Keep records updated with completed work, inventory changes, and property conditions.

7. Financial Operations: Manage all aspects of payment processing.

8. Third-Party Integrations: Seamlessly connect with external systems.

These categories help companies:

• Assess their current level of management automation.

• Identify gaps in their autonomous capabilities.

• Plan development roadmaps to increase automation.

• Benchmark against industry standards.

• Validate systems against real-world edge cases.

Each category includes detailed criteria to ensure systems can handle tasks without human intervention. While it’s a comprehensive list, the idea is to aim for most of these capabilities—similar to a typical human performance. Achieving all could mean the system outperforms most humans.

Current v1 Benchmark Results

Here’s a snapshot of how vendors have scored so far, in the v1 of this benchmark:

Good documentation and proactive steps; struggles with communication and edge case management

How You Can Get Involved

• Submit Your Vendor: If your system automates cleaning and maintenance management, please submit a PR. Your submission should include your system’s name, a description, and a demo or another form of validation.

• Improve the Checklist: We welcome improvements and suggestions. If you have ideas to make the checklist more specific or relevant, create a PR with your changes.

The open source repository is here: https://github.com/TIDYAPP/maintenance-management-automation-benchmark

Final Thoughts

The Autonomous Cleaning and Maintenance Management Benchmark is designed to be a flexible and comprehensive tool for anyone looking to push the limits of automation in this industry. By open-sourcing this project, we hope to foster collaboration and accelerate innovation.

Join us in shaping the future of cleaning and maintenance management. Submit your vendor information, share your feedback, and help improve the benchmark.

Let’s work together to drive the next generation of autonomous management systems.

Feel free to reach out with any questions or suggestions. We look forward to your contributions!

‍