Lab experimentFirst experiment: Factory as the harness
Experimenting with how we build construction software.
ConTechLabs explores how AI can help AEC professionals understand requirements, resolve constraints, reduce avoidable cost, communicate decisions, and learn from completed work. The Software Factory examines how we build and improve the applications that support that work.
We’re investigating a development process in which AI performs much of the analysis, implementation, testing, and review preparation. People define the purpose, apply construction and engineering judgment, assess trade-offs, and decide which changes should proceed.
A Software Factory brings those activities into a repeatable workflow: a useful signal becomes a defined task, a reviewable change, a verified release, and evidence for the next improvement. We want to learn which methods and model configurations improve the complete process, including the effort required to supervise and correct the work.
The questionCan we turn valid and early signal into useful construction software improvements with less delivery time and total effort, while maintaining quality, traceability, and human control?
01 / The lab experiments on its own methods
Improve the applications. Improve how we deliver them.
Learn from construction work.
Our construction experiments investigate blockers, lead times, cost, communication, and learning. Their findings should help define concrete software behavior: which records to connect, which questions to investigate, which decisions to prepare, and how to establish whether the result helped.
For an AEC professional, the outcome should be a useful application that understands the work and produces a result they can inspect and act on.
Learn from software delivery.
The Software Factory tests the development approach behind those applications. We would examine how agents receive context, prepare changes, work with people, verify behavior, and respond to production feedback.
The two loops inform each other: construction experience improves the product requirements, and delivery evidence improves the methods used to implement them.
Valid and early signal applies here too. A user report, test failure, or production error should arrive with enough evidence and context for the team to decide what to investigate or change while the response is still useful.
02 / What we are exploring
The method matters alongside the model.
We would evaluate the whole development workflow, from understanding the request through observing the released software. A model’s output is one part of that result. Context, tools, task definition, coordination, and verification also need to be assessed.
Specify the outcome.
Describe the user need, construction context, expected behavior, and acceptance criteria before implementation. Keep the originating evidence linked to the task.
Give agents working context.
Provide repository instructions, relevant specifications, test data, and permitted tools. The harness is the working environment that coordinates context, model calls, tools, and execution.
Evaluate models in the workflow.
Compare model-and-harness configurations on representative development tasks. Record the model, instructions, tools, cost, and review effort so results can be interpreted.
Coordinate the work.
Explore individual sessions, planned missions, and coordinated agent work for tasks that benefit from them. Make ownership, dependencies, and integration work visible.
Verify the change.
Check actual behavior with tests, code review, security review, and preview QA. Compare the result with the accepted task and evaluate effects on existing functionality.
Learn from delivery.
Connect released changes to user feedback, errors, and observed behavior. Use verified results to propose improvements to the software, instructions, skills, and evaluations.
03 / Software experiment 01
Use Factory as the harness.
Our first experiment will use Factory to provide the environment in which AI agents work with code, repository context, models, and connected tools. Factory’s development agents are called Droids.
We plan to evaluate work started by a person alongside work proposed in response to incoming signals. Both paths should produce a defined scope, traceable changes, and evidence that the result meets the agreed requirements.
Experiment design: The workflow below describes what we intend to evaluate. The listed sources are in scope; the diagram does not establish that connections or automations are already configured. People approve scope, plans, pull requests, and releases.
Signals in scope
Slack
Development discussions, reported problems, and clarification.
Linear
Tasks, ownership, requirements, and acceptance criteria.
GitHub
Code, issues, pull requests, reviews, and change history.
Sentry
Reported errors and diagnostic context.
Notion
Specifications, decisions, and working documentation.
Email
Relevant feedback and requests shared with the workflow.
CI
Automated build results, tests, and failing checks.
People
Goals, ideas, domain expertise, judgment, and approval.
People-initiated work
A person defines the task.
A ticket, conversation, reported problem, or idea becomes a concrete task. Factory Droid uses the relevant context to prepare a plan, implement the approved work, and produce a pull request for review.
Signal-initiated work
Agents prepare the next useful action.
An agent assesses incoming evidence, connects related records, and prepares a proposed task or clarification. Following scope and plan approval, Droids prepare the implementation and linked pull request.
04 / From signal to production feedback
One connected development loop.
This diagram uses Claude Code as the harness. Other options, such as Factory Droid, Warp, or Codex, could also be used. We may experiment with different harnesses and models as the lab progresses, comparing their delivery time, total effort, quality, and observed results.
People set the sources, goals, ownership, tools, review requirements, and permitted autonomy for each task. Both development paths pass through code and security review, behavior-change review, independent risk review, tests, CI checks, and preview QA before release approval.
People set the rules
Sources and access
Goals and owners
Skills and tools
Review and release gates
Autonomy by task
Signals
Triage
Build
Validate
Ship and observe
Signals
Slack
Linear
GitHub
Sentry
Notion
Email
CI
People
People-initiated work
A person defines the task
A ticket, thread, alert, or idea becomes a defined task.
Build with a Claude Code session
Use repository context and connected tools. Prepare a reviewable pull request.
People provide goals, context, and judgment.
Signal-initiated work
Agents assess the signal
Check evidence, relevance, duplication, and the next useful action.
A Claude Code session prepares the change
Prepare the plan, implement the approved scope, and open a linked pull request.
A person approves scope, the plan, and the pull request.
Checks for each pull request
Code and security review
Behavior-change review
Independent risk review
Tests and CI checks
Agent QA on a preview
A person approves.
Ship and observe
Merge after review.
Release. Validate in preview or staging. Release when approved.
Observe and trace back. Connect errors, user reports, and release evidence to the changes that may explain them.
↻ What production shows becomes the next signal.
Software experiment 01: Claude Code as the harness. Proposed workflow; connections and automations require configuration.
01
Signals
Gather permitted records and observations. Retain the source, date, affected application, and uncertainty.
02
Triage
Check relevance, duplicates, and evidence. Prepare a task, a targeted question, or a finding that no change is needed.
03
Build
Use a Claude Code session to prepare a plan and implement the approved scope. Keep the task, code changes, and pull request connected.
04
Validate
Review code, security, behavior changes, and risk. Run tests and CI checks, exercise the preview, and obtain the responsible person’s approval.
05
Ship and observe
Merge after review, validate the release, and publish when approved. Investigate production evidence and trace it to relevant changes.
A passing build, an AI review, a human approval, and a verified production outcome are separate events. Preserve the evidence for each one. Production findings inform the next task; an agent’s proposed explanation remains open to investigation.
05 / An illustrative lab task
A coordination prototype cites a superseded drawing.
A user reports that an experimental application included an older drawing reference in its coordination summary. The report becomes a signal to investigate the document selection and source references.
Prepare a correction the team can verify.
An agent assembles the report, relevant requirements, source revisions, and code history. It prepares a proposed change and a regression test using the revised drawing input.
Following approval, Factory Droid implements the change. Reviewers inspect the behavior change, test results, and preview output. An authorized release is then checked against the reported problem.
The result informs the application’s document-handling rules and future tests. The software change does not authorize a construction design change or issue project direction.
06 / Evidence of value
Measure the complete delivery process.
We would compare representative tasks with similar scope and acceptance criteria, retaining the model, harness configuration, instructions, and tool versions used. Where practical, change one major variable at a time and examine the resulting differences.
Include failed attempts, review effort, and post-release corrections. Faster code generation is useful when it contributes to a better overall delivery result.
Time from a supported signal to an accepted task and a verified release.
Human effort spent clarifying, supervising, reviewing, and correcting work.
Model, tool, and execution cost, including unsuccessful attempts.
Whether the change meets the user need and preserves existing behavior.
Defects, regressions, security findings, and issues discovered after release.
Traceability from the original evidence through the decision, pull request, tests, and production result.
Unsupported alerts, unnecessary changes, and useful signals the workflow missed.
Improvements that remain useful when applied to subsequent tasks.
Factory documents connected tools, event- and schedule-triggered automations, and structured agent workflows. The review sequence and operating practices described on this page are ConTechLabs’ proposed experiment. Feature availability and connections must be checked during setup.