The OL definitely played a big part. Hiring Bill Callahan and attacking the position group in the draft and FA will solve that.
The bigger part IMO was the coaching. It's an open secret that Ken Zampese was the brains behind the success of the 2018 passing offense - both in terms of coaching up Baker mechanically and in designing the base and 3rd down passing offense. Freddie Kitchens felt threatened, and after getting promoted to HC on the backs of his offensive assistants, promptly fired Zampese and the OL coach - Bob Wylie - who had the Browns OL ranked 2nd overall by PFF that year. He replaced Zampese with Ryan Lindley, who had never been a QB coach before.
Promoting Kitchens to HC was the beginning of the end for Baker and the Browns in 2019. It takes a unique man to get fired at HC after only one year on the job. Kitchens goes down in history as one of the few to actually pull it off.