Could you elaborate on what you mean about the development of deep learning architecture in recent years?

Transformers. Fun fact, the T in GPT and BERT stands for "transformer". They are a neural network architecture that was first proposed in 2017 (or 2014 depending on how you want to measure). Their key novelty is the method of implementing an attention mechanism and a context window without recursion, which was the method most earlier NNs used for that.

The wiki page I linked above is admittedly a bit technical, this articles explanation might be a bit more friendly to the layperson.