Joby plans to expand way beyond airport access, it’s meant to be basically flying rideshare. The key enabler is they designed it to be quiet enough to not annoy everyone around like a helicopter, so that it would be reasonable to have this thing taking off from residential neighborhoods. JFK access is just a very visible first test run.
This thing will not be taking off from residential neighborhoods. Regardless of noise, like any manned VTOL aircraft it requires a large open area free of overhead obstructions (trees, wires).
And the notion that landing pads will be installed on the roofs of tall buildings is mostly a fantasy. Commercial or high-rise roofs are mostly already occupied with other machinery, and aren't designed to support the extra weight.