It should go without saying at this point that since VoIP is sending your voice as data packets over the internet, it will be directly impacted by the latency on your network.
This means that VoIP will be directly impacted by latency due to congestion, a lack of bandwidth to handle traffic, or restricting hardware and software configuration.
With higher latency in your network, the chances of experiencing jitter is a lot higher. Due to a lower bandwidth, and slower download/upload speeds, your network will be able to handle less consecutive actions before slowing down.