Add adaptive concurrency controller in preparation for adaptive inference#1784

jgreer013 merged 8 commits intomainoumi-ai/oumi:mainfrom

jgreer013/adaptive-inferenceoumi-ai/oumi:jgreer013/adaptive-inferenceCopy head branch name to clipboard