Spoken Dialogue System (SDS) utilize on a sequential processing pipelines that introduce compounding latency across Automatic Speech Recognition (ASR), Large Language Model (LLM), and Text-to-Speech (TTS) components, making natural real-time voice interactions difficult to achieve. This study evaluates how the choice of a LLM and turn-taking strategy affect the SDS latency when it is optimized for cold calling purposes. Together, four LLMs were tested under controlled conditions, which were GPT-4o-mini, GPT-5-Nano, Gemini-2.5-Flash-Lite, and GPT-5. Furthermore, three turn-taking strategies were also tested, specifically: Conservative, Rule-based, and Neural network. The results show that GPT-4o-mini produced the lowest and most consistent latency with a mean of 497.7 ms, while GPT-5 reached a mean of 1673.1 ms despite being a newer model and showing relatively consistent response times. For turn-taking, the Neural network strategy achieved the lowest mean Turn Latency at 2087 ms and notably reduced LLM Latency to 336 ms, partly attributed to AssemblyAI’s speaker classification output providing a cleaner conversational context to the language model. No component configuration successfully managed to bring down the end-to-end latency within the 200 ms threshold humans associate with natural conversation. This confirms that sequential pipeline architecture remains the primary structural bottleneck regardless of the performed component level optimization within the pipeline.