INSIGHTS / DEVICE SUPPORT
Which device-support tasks should you automate first?
Download the editable audit workbook (.xlsx)
Includes Guide, Audit, Evidence and Example sheets. No signup required.
For a device-company owner or support manager, the first AI decision is where to put a limited pilot budget. Start with a recurring task that consumes meaningful staff time and has a reliable procedure, accessible evidence and a result you can check. High ticket volume alone is not enough.
For a connected-device vendor or systems integrator, compare workload with readiness for the specific task. Then choose the smallest useful pilot whose result you can verify. The accompanying workbook gives your support team ten starting categories, a workload calculation and a completed fictional example to work through with you. Adapt the categories to your products.
1. Count cases and active work consistently
Choose one complete reporting period and record its dates. Group related messages and reopened conversations into the same underlying case where they concern the same unresolved issue. Give each case one primary category so it is counted once.
Record the case count and average active handling minutes for each category. Count time spent diagnosing, documenting, chasing information and handling reopens. Exclude elapsed time when nobody is working on the case. Record a cutoff date for follow-up work; cases still open at that date have incomplete handling time. If time comes from a sample, record its size and selection method. Leave unknowns blank. The workbook flags incomplete or invalid inputs and totals only rows marked Ready; treat that subtotal as partial until all relevant categories are complete.
The workbook calculates:
Handling hours = cases × average active minutes per case ÷ 60.
For illustration, 80 cases at 12 minutes each represent 16 hours of work. Use this baseline to judge whether a bounded task deserves the cost of a pilot and ongoing review. It is not a savings forecast. If quick information requests and lengthy repairs share one label, split them before calculating an average.
2. Audit the task behind the ticket
Use these ten categories as a starting point. Replace irrelevant categories and split broad ones when their procedures differ. The workbook includes ten spare rows, already linked to Evidence and included in its totals. Keep the rows in place and rename categories; it supports up to twenty categories.
| Ticket type | What to establish before a pilot |
|---|---|
| Device appears offline | Use for absent or unknown heartbeat/check-in. Establish whether evidence distinguishes a stopped device from a reporting problem. |
| Delayed or missing readings | Use when check-ins continue but payload data is stale or absent. Check reporting interval, clock alignment and destination receipt. |
| Installation or onboarding | Exact model/version, prerequisites and an approved procedure for checking setup. |
| Login or account access | Identity checks and an approved access-recovery path; no improvised bypass. |
| Configuration mismatch | Authoritative target settings, current settings and who may approve a change. |
| Power or battery complaint | Safe observations to collect and the boundary for trained human inspection. |
| Connectivity or SIM issue | Which network, account and device facts are available and who owns the next diagnostic step. |
| Firmware or software update | Compatibility, approved procedure, recovery path and change authority. |
| Unexpected alert | The rule, event time and device context needed to interpret the alert. |
| Replacement or warranty request | Relevant policy, purchase/device record and the person authorised to decide. |
“Device offline” alone is too broad a pilot task. “Collect the model, last successful report time and approved connectivity checks, then prepare a handover” is a bounded task. It may be useful even when the AI cannot repair the device.
A timestamp also needs a definition. In a fictional sequence, the device measures power at 09:10, the platform receives that report at 09:11, and the AI retrieves it at 11:00. Another retrieval at 11:03 still describes the 09:10 observation. It cannot establish that power is present now. Record event time, receipt time and retrieval time separately where available; do not substitute one for another.
AWS IoT illustrates this distinction: its shadow document separates the response timestamp from metadata showing when individual state attributes were updated. Those platform timestamps still need to be interpreted according to what they measure. [3]
3. Choose the next step and its test
The Audit sheet records workload and the proposed next step. The Evidence sheet holds the procedure/version, data requirements, completion test, authority, gap and reviewer. The separate sheets keep workload comparisons compact while leaving room for technical evidence. Match the stable category reference (T01–T20) across both sheets. These are workbook row references, not customer ticket IDs. Use evidence references rather than copying customer messages or customer, device or account identifiers into the workbook.
Choose for the specific task you intend to test. Use the least autonomous option that can deliver the useful output; workload alone does not decide it.
Here, automation readiness means the proposed task has its required procedure, evidence, authority and completion test. The workbook's Ready flag has a narrower meaning: the workload inputs are complete and valid enough to include in the subtotal. It does not certify that the task is ready for automation.
| Next step | When to choose it | What to verify |
|---|---|---|
| Fix prerequisites | The task depends on unavailable or untrusted data, a missing procedure or undefined authority. | The named gap is resolved and verified before testing the dependent task. |
| Assist a person | AI drafts a summary or suggestion; a person reviews every output before it is used. | Correct facts, explicit unknowns and the review/correction time. |
| Test read-only triage | AI classifies or routes cases without a person reviewing every output before use; it cannot change device state. | Correct routing and handover, including missing/conflicting evidence and urgent cases. |
| Test approved action | A specific state-changing action has explicit authority, preconditions and a recovery or escalation path. | Permission checks, refusal outside scope and the actual post-action result. |
These choices define a pilot scope, not permission to deploy. Define success for that scope: a handover test for a summary task, a routing test for triage, or a device-state test for an approved change.
To choose between candidates, first set aside tasks whose own prerequisites are missing, or narrow them to an assist task that can proceed safely. Among candidates that pass those checks, compare the active work attributable to the proposed task, the effort to set up and review the pilot, and the consequences of an error. Prefer a useful task with a clear completion test and manageable review or recovery. If two candidates are otherwise similar, the one consuming more active staff time is the stronger candidate to test. This is a practical selection rule, not a validated scoring model or a savings forecast. Do not count a whole ticket's handling time as time the proposed task could eliminate.
As a product-specific illustration of prerequisites, Atera's AI-assistant troubleshooting guide checks enabled actions, account integration, device assignment and connectivity when its assistant cannot act. [1]
4. Work through one candidate
The table below summarises the workbook's fictional offline-device example. The Example sheet also includes a reporting period, follow-up cutoff and reviewer role; replace these with your own dates and named reviewer:
| Field | Completed example |
|---|---|
| Workload | 80 cases; 12 active minutes per case; 16 hours in the selected period. |
| Procedure | Fictional guide D-01, version 3: collect model, symptom and permitted connectivity observations. |
| Available data | Model lookup is available. Last-seen timestamp is unreliable and must be marked unknown. |
| Proposed task | Draft an intake summary for technician review. |
| Completion test | Summary identifies the correct device/model, symptom and observations with their sources; explicitly marks unavailable facts; makes no unsupported repair claim. |
| Authority | Retrieve and summarise permitted records only. No reset or configuration change. |
| Next step | Assist a person. Intake can proceed with the timestamp marked unknown. |
| Gap to fix | Verify the timestamp source before using it to diagnose connectivity or confirm restoration. |
The distinction matters: the missing timestamp blocks a diagnosis that depends on it. It does not prevent a useful summary that states what is unknown.
5. Decide what would justify the next step
For the intake task, test records with a missing device ID, conflicting model information, a plausible-looking timestamp explicitly labelled untrusted, and a reset request outside scope. Specify the correct summary or handover for each. Use redacted cases and keep final assessment cases separate from those used for tuning.
Measure wrong facts, omitted unknowns, prohibited actions, review time and rework. Set acceptance criteria before testing. Anthropic's evaluation guidance distinguishes an agent's transcript from the actual outcome and defines tasks and graders explicitly. [2]
For a time comparison, measure the same bounded task on comparable cases under the current process and the pilot. Include human review and correction in the pilot's active handling time. A faster draft is not a benefit if checking it consumes the time saved or creates more errors.
At Agentra, this audit turns domain knowledge into a testable support task. Complete Audit and Evidence for each shortlisted category, then use the selection rule above to choose the first bounded pilot.
Related reading: Agentra device support · AI agent action controls
Sources
- Atera: Troubleshoot the AI portal assistant. Rechecked 11 October 2026. Product-specific example of operational prerequisites.
- Anthropic: Demystifying evals for AI agents. Rechecked 11 October 2026. Evaluation design and outcome verification.
- AWS IoT: Device Shadow service documents. Rechecked 11 October 2026. Response time versus per-attribute update metadata.
The workbook, ticket taxonomy and fictional example are Agentra's proposed audit method. They are not a validated scoring model or reported customer results.
