Major hyperparameters which affect fuzzy time series (FTS) forecasting are the number of partitions, length of partition intervals in the universe of discourse, and the fuzzy order. There are very few studies which have considered an integrated solution to optimize all the hyperparameters. In this paper, we strive to achieve optimum values of all three hyperparameters for fuzzy time series forecasting of the COVID-19 pandemic using the Particle Swarm Optimization (PSO) algorithm. We specifically propose two techniques, namely nested FTS-PSO and exhaustive search FTS-PSO for determining the optimal interval length, as an augmentation to the FTS-PSO model that optimizes the interval length and the fuzzy order. Nested PSO has two PSO loops: (i) the inner PSO optimizes the combination of fuzzy order and boundaries of intervals for a given number of partitions defined by the outer loop, and the resultant cost is fed back to the outer PSO; (ii) the outer PSO optimizes the number of partitions to reduce the cost while meeting the defined constraint. Exhaustive search FTS-PSO also has two loops where the inner loop is similar to nested FTS-PSO while the outer loop iterates over a pre-defined search space of number of partitions. We analyze the effectiveness of the two approaches by comparing with ARIMA, FbProphet, and the state-of-the-art FTS and FTS-PSO models. We adopt COVID-19 highly affected 10 countries worldwide to perform forecasting of coronavirus confirmed cases. We consider two phases of COVID-19 spread, one from the year 2020 and another from 2021. Our study provides an analytical aspect of the COVID-19 pandemic, and aims to achieve optimal number and length of intervals along with fuzzy order for FTS forecasting of COVID-19. The results prove that the exhaustive search FTS-PSO outperformed all the methods whereas nested FTS-PSO performed moderately well.