For years and years, Google has been trying to communicate that 404s, page not found results, are a normal part of the web. Pages don't always live forever. 404s are not a bad thing, they are a normal thing. Of course, you don't want to return a 404 for a page that should exist, but a normal 404 is, um, normal.
That being said, John Mueller once again said, 404s are not a signal of having a low quality site.
This statement came on Twitter this morning in response to that question. Here it is:
@wockenfoth no, that's definitely not the case.— John ☆.o(≧▽≦)o.☆ (@JohnMu) March 17, 2017
40x status codes are not a sign of a low quality site, it just means there are certain URLs on your site that do not exist and that is okay.
Forum discussion at Twitter.