I can remember - I don't want to identify the individual - but a very prominent Democrat, who compared looking at Carter and then Reagan, and then Bush, and observed that many of the people around Carter were totally disloyal to him.
The standard rumor at the time was that Rumsfeld, as chief of staff, had persuaded President Ford to appoint George H.W. Bush as director of Central Intelligence, assuming that that got rid of a potential competitor for the presidency.
Another factor is the decision, made in 1976, to sharply divide the FBI and the foreign intelligence agencies. The FBI would collect within the United States; the foreign intelligence agencies would collect overseas.
And Oliver North was really a good soldier, up to the last moment, shoving memos into the shredder and defending the policy to the end.