r/LocalLLaMA · · 1 min read

I created a personality test for models, need more TESTS!!

Mirrored from r/LocalLLaMA for archival readability. Support the source by reading on the original site.

I created a personality test for models, need more TESTS!!

First 3 disposition results.

Hi there!

My name is Jerry and I recently built Enclosure, e deterministic environment for testing LLMs. It's a simulated office enfironment https://github.com/openconstruct/Enclosure with tools agents are used to, like slack, calendar, mail, chat and more. For conversations is uses AIML instead of a model, so it is totally deterministic.

The first test I made(had Claude make) is the one I had in mind when I designed Enclosure, a personality test for models. It's not a winnable benchmark, but rather a tool to help people find the mdoels that fit their workstyle best. It's called disposition and you can find it here: https://github.com/openconstruct/disposition

I tested the cheapest models on Alibab modelstudio already, after I refill my opencode go next month, I will probably test more. I am asking the community to test some models if you think it's a cool project.

Instructions are in the Disposition repo, and the submission repo is here: https://github.com/openconstruct/disposition

submitted by /u/thebadslime
[link] [comments]

Discussion (0)

Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.

Sign in →

No comments yet. Sign in and be the first to say something.

More from r/LocalLLaMA