In early Mixture-of-Experts models, individual experts were relatively interpretable and showed c..., Sonic AI
“In early Mixture-of-Experts models, individual experts were relatively interpretable and showed clear specialization on concepts like dates or specific object types.”