With Sunday’s loss to a previously winless Jacksonville Jaguars team, Andrew Luck and the Indianapolis Colts fell into a three-way tie for last place in the AFC South. The division is arguably the worst in the NFL and has been for the better part of Luck’s career with the Colts.

When people wonder what’s wrong with the Colts the first person, and usually the easiest, is to blame the quarterback, but the reality is he’s just about the only thing going right for the Colts. The reality is Andrew Luck has been masking the Colts deficiencies for a while and we’re only starting to see the cracks in the foundation.

Bad supporting casts around a generational quarterback talent is nothing new to the Colts organization. After the death of his father, Bill, Jim Irsay became the youngest owner in the NFL in 1997 at age 37. The fortune he has received in those nearly two decades of his ownership is nothing short of miraculous.

Finding a franchise quarterback in the NFL is no small task. There’s no exact science to finding one. Quarterbacks come from all walks of life. They can be found at the very top of the draft, born of football pedigree, or in the middle of the draft out of small, midwestern mid-major schools or even as undrafted free agents, but there’s no guarantee that they are found. Some franchises look for years, even decades for one. It can even be argued that some franchises have never even really had one at all.

Then there is a team like the Colts. In 1997, the Colts went 3-13 and were awarded the top pick in the 1998 NFL Draft, where two major quarterback prospects coming available, Ryan Leaf from Washington State and Peyton Manning out of Tennessee. Team scouts preferred Leaf but then-Colts General Manager Bill Polian preferred Manning. Polian won and the rest with Manning is history. He went on to have a stellar career which will likely end with him being enshrined in the Hall of Fame with the Colts while Leaf has become arguably the biggest bust in NFL history.

For the next 13 seasons, Manning started every game for the franchise, won four league MVP awards and Super Bowl XLI. However, in the spring of 2011, Manning had a neck surgery that was eventually followed by a spinal fusion surgery in September. Suddenly, the quarterback who started 208 consecutive regular season games missed the entire season and would never play another down as an Indianapolis Colt.

In Manning’s absence, the Colts struggled, going 2-14 and securing the first pick in the 2012 NFL Draft. With one of the most highly touted draft prospects in years coming out in Stanford’s Andrew Luck, the Colts had decided to tear it down and rebuild. Both Polian and Head Coach Jim Caldwell were let go, ultimately replaced by Ryan Grigson and Chuck Pagano. On March 7, 2012 Manning was released. That April, the Colts selected Luck with the top pick of the draft, quickly ushering in a new era in Indianapolis.

Over the next three years, Luck would prove exactly why he was so coveted. He took a 2-14 team that was (and still is) pretty talent deficient and took them to the playoffs for three consecutive years culminating in a 2014 season where he threw for 4,761 yards and 40 touchdowns en route to an AFC Championship Game appearance. He also ridiculously lost the Offensive Rookie of the Year award in 2012 to Robert Griffin III, who took a far more capable team to the same lengths and didn’t look nearly as impressive doing it.

That said, most, if not all, of the Colts success is tied to Luck. For basically the second time in his short career with the team they seem to be rebuilding the offensive line. Just this weekend they started three rookies on the line with inevitably poor results. They haven’t had a running back pick up 100 rushing yards in a game since 2013. They have one good wide receiver in T.Y. Hilton and a mixed bag behind him. A washed up Andre Johnson was signed last year only for it to fail miserably. And all these are just issues with the offense, an offense that is still scoring 27.0 points per game. The major issues lie more with the defense.

The defense boasts one elite level player, cornerback Vontae Davis. One of the very few acquisitions Grigson actually got right in his tenure as general manager. The rest of defense is quite uninspiring. They’re regularly beaten through the air and on the ground and rarely seem to make a stop when needed. And on the occasions they share the field with the New England Patriots, they always seem to get outplayed in all aspects of the game.

Grigson has failed as a talent evaluator. Of his five years with the Colts he’s made 38 draft picks. Of those 38, only 18 are still with the team, a number skewed by the fact that all eight of this year’s draft picks made the roster. Otherwise, its 10 of 30, which is an alarming failure rate. That also doesn’t include his trade of a first round pick for former Cleveland Browns running back Trent Richardson, who proved to be an awful acquisition as well.

Following their 2014 AFC Championship Game loss to the Patriots by a final score of 45-7, it was clear to everyone that the Colts needed some serious help on defense. So with their first pick in the 2015 NFL Draft, naturally they drafted Phillip Dorsett, a small wide receiver from the University of Miami. To Dorsett’s credit he was a fine prospect with some game-breaking potential thanks to his 4.33 40-yard dash time, but, in theory, his ceiling as a player was that of Hilton, who is already on the team. They would have been better served addressing the defensive side of the ball, especially with cornerback Ronald Darby, who finished second in voting for Defensive Rookie of the Year still available. It’s shortsighted moves like that that make Grigson look incompetent,

Pagano hasn’t fared much better. When Pagano took a leave of absence in 2012 after he was diagnosed with leukemia, then Offensive Coordinator Bruce Arians took over and helped lead them to the playoffs. After the season, Arians was hired as the head coach of the Arizona Cardinals, who have gone 36-17 since. Pagano came in as defensive coach, previously the defensive coordinator of the Ravens, and the Colts have not had much of a defense.

If for no other reason, how could a good or competent coach run a play like this:

So who’s really to blame for the Colts struggles? Seems pretty obvious the answer is the coach and the general manager. Indianapolis is lucky to have a franchise quarterback, but they’re doing themselves and their fans a disservice by not surrounding him with the necessary talent to succeed and take them to where they really want to go. Football fans watched the Dolphins struggle to surround Dan Marino with enough talent to win it all. He carried them all the way to Super Bowl XIX in 1984, where they lost to the San Francisco 49ers. It was only Marino’s second year. People were sure his titles were to come. Yet, they never did. Hopefully for Luck, his 2014 AFC Championship game appearance won’t be his Super Bowl XIX.