The Vicsek fractals are one of the most interesting classes of fractals and the study of their structural properties is important. In this paper, the exact formula for the mean geodesic distance of Vicsek fractals is found. The quantity is computed precisely through the recurrence relations derived from the self-similar structure of the fractals considered. The obtained exact solution exhibits that the mean geodesic distance approximately increases as an exponential function of the number of nodes, with the exponent equal to the reciprocal of the fractal dimension. The closed-form solution is confirmed by extensive numerical calculations.PACS numbers: 83.80. Rs, 61.43.Hv The concept of fractals plays an important role in characterizing the features of complex systems in nature, since many objects in the real world can be modeled by fractals [1]. In the last two decades, a great deal of activity has been concentrated on the studies of fractals [2,3]. It has been shown [4,5,6,7,8,9,10,11] [26,27,28,29,30,31].Despite the importance of this structural property, to the best of our knowledge, the rigorous computation for the mean geodesic distance of Vicsek fractals has not been addressed. To fill this gap, in this present paper we investigate this interesting quantity analytically. We derive an exact formula for the mean geodesic distance characterizing the Vicsek fractals. The analytic method is on the basis of an algebraic iterative procedure obtained from the self-similar structure of Vicsek fractals. The obtained precise result shows that the mean geodesic distance exponentially with the number of nodes. Our research opens the way to theoretically study the mean geodesic distance of regular fractals and deterministic networks [32,33,34]. In particularly, our exact solution gives insight different from that afforded by the approximate solution of stochastic fractals.The classical Vicsek fractals are constructed iteratively [12,15]. We denote by V f,t (t ≥ 0, f ≥ 2) the Vicsek fractals after t generations. The construction starts from (t = 0) a star-like cluster consist of f + 1 nodes arranged in a cross-wise pattern, where f peripheral nodes are connected to a central node. This corresponds to V f,0 . For t ≥ 1, V f,t is obtained from V f,t−1 . To obtain V f,1 , we generate f replicas of V f,0 and arrange them around the periphery of the original V f,0 , then we connect the central structure by f additional links to the corner copy structures. These replication and connection steps are repeated t times, with the needed Vicsek fractals obtained in the limit t → ∞, whose fractal dimension is ln(f +1) ln 3 . In Fig. 1, we show schematically the structure of V 3,2 . According to the construction algorithm, at each time step the number of nodes in the systems increase by a factor of f + 1, thus, we can easily know that the total number of nodes (network order) of V f,t is N t = (f + 1) t+1 . After introducing the Vicsek fractals, we now investi-

No abstract

The Off-Policy Evaluation (OPE) aims at estimating the performance of target policy π using offline data rolled in by a logging policy µ. Intensive studies have been conducted and the recent marginalized importance sampling (MIS) achieves the sample efficiency for OPE. However, it is rarely known if uniform convergence guarantees in OPE can be obtained efficiently. In this paper, we consider this new question and reveal the comprehensive relationship between OPE and offline learning for the first time.For the global policy class, by using the fully model-based OPE estimator, our best result is able to achieve ǫ-uniform convergence with complexity O(H 3 • min(S, H)/d m ǫ 2 ), where d m is a instance-dependent quantity decided by µ. This result is only one factor away from our uniform convergence lower bound up to a logarithmic factor. For the local policy class, ǫ-uniform convergence is achieved with the optimal complexity O(H 3 /d m ǫ 2 ) in the off-policy setting. This result complements the work of sparse model-based planning Agarwal et al. ( 2019) with generative model. Lastly, one interesting corollary of our intermediate result implies a refined analysis over simulation lemma.

We consider the problem of offline reinforcement learning (RL) -a well-motivated setting of RL that aims at policy optimization using only historical data. Despite its wide applicability, theoretical understandings of offline RL, such as its optimal sample complexity, remain largely open even in basic settings such as tabular Markov Decision Processes (MDPs). In this paper, we propose Off-Policy Double Variance Reduction (OPDVR), a new variance reduction based algorithm for offline RL. Our main result shows that OPDVR provably identifies an -optimal policy with O(H 2 /d m 2 ) episodes of offline data in the finite-horizon stationary transition setting, where H is the horizon length and d m is the minimal marginal state-action distribution induced by the behavior policy. This improves over the best known upper bound by a factor of H. Moreover, we establish an information-theoretic lower bound of Ω(H 2 /d m 2 ) which certifies that OPDVR is optimal up to logarithmic factors. Lastly, we show that OPDVR also achieves rate-optimal sample complexity under alternative settings such as the finite-horizon MDPs with non-stationary transitions and the infinite horizon MDPs with discounted rewards.

No abstract