Selssabil·Oct 4, 2024Exploring SwiGLU : The Activation Function Powering Modern LLMsI was looking into the LLaMA architecture and came across three key modifications to the original Transformer that LLaMA introduces. One…A response icon3A response icon3