They would just be removed from the line up succession. We’ve had cabinet members who were ineligible for the presidency, and they were just not considered as in the line of succession, even though they otherwise would be.

Edit: here’s the text of the Presidential Succession Act of 1947

(b) If, at the time when under subsection (a) a Speaker is to begin the discharge of the powers and duties of the office of President, there is no Speaker, or the Speaker fails to qualify as Acting President, then the President pro tempore of the Senate shall, upon his resignation as President pro tempore and as Senator, act as President.

Note that it explicitly mentions the Speaker failing to qualify for the presidency