[–] 2 points 5 days ago (1 child)

I don't know that paper and am not an expert in model architecture. But to my understanding they are still predictive. An important improvement beyond adding more parameters was the introduction of MoE (mixture of experts?) architecture. Basically this segregates the models into experts that are only activated when needed.

  • source
  • parent
  • context
  • view more: next ›