Best practices when running a benchmark on online models [D]
Mirrored from r/MachineLearning for archival readability. Support the source by reading on the original site.
I'm developing a benchmark for a low resource language and I don't want it to be leaked and used for training when it is being used to get predictions. For locally run models it shouldn't be a problem, but for models that are only accessible via API, it is. Is there an established way to evaluate online models without the input data being lost? Do you trust Google and OpenAI when they say that they do not use your inputs for training when you have a paid account?
[link] [comments]
More from r/MachineLearning
-
I turned my dog Jackies Brain into a working AI [P]
Oct 14
-
Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]
Oct 14
-
Binary noise classification using CRNN mean teacher algo [R]
Oct 14
-
Localized Neural learning vs backpropagation [D]
Oct 14
Discussion (0)
Sign in to join the discussion. Free account, 30 seconds — email code or GitHub.
Sign in →No comments yet. Sign in and be the first to say something.