Kiodai
can ai agents actually remember to do something later?
read the paper (PDF, 11 pages)
i wrote Kiodai to study prospective memory in ai agents: whether they can remember an instruction and carry it out later, even when deadlines change, tasks get cancelled, or other conditions need to be met first.
i tested three approaches on DeepSeek V3.1: a baseline without external memory (A0), a structured memory ledger the agent could query (B_ledger), and a structured store with programmatic monitoring (A2). i compared them across 12 synthetic task sequences from four template families, with one run per setup and sequence.
the results weren't what i expected.
the baseline scored 0.8643 in mean trajectory Set-F1, compared with 0.8111 for the ledger and 0.4806 for the monitored system. the baseline also had the lowest API-reported cost. my hypothesis that structured memory would improve performance wasn't supported.
looking through the logs, i found old deadlines resurfacing, actions linked to the wrong instructions, and cases where the agent had the information it needed but still made the wrong decision.
the main thing i learned was that storing an instruction doesn't mean an agent will understand or execute it correctly.
this was a small experiment using one model configuration, synthetic scenarios already seen during development, and no repeated runs. it doesn't prove structured memory is generally ineffective, and i couldn't isolate exactly why the more complex systems performed worse.