A two-timescale simulation-based actor-critic algorithm for solution of infinite horizon Markov decision processes with finite state and compact action spaces under the discounted cost criterion is proposed. The algorithm does gradient search on the slower timescale in the space of deterministic policies and uses simultaneous perturbation stochastic approximation-based estimates. On the faster scale, the value function corresponding to a given stationary policy is updated and averaged over a fixed number of epochs (for enhanced performance). The proof of convergence to a locally optimal policy is presented. Finally, numerical experiments using the proposed algorithm on flow control in a bottleneck link using a continuous time queueing model are shown.

No abstract

Popularity of herbal drugs is increasing all over the world because of lesser side effects as compared to synthetic drugs besides it cost effectiveness and easy availability to poor people particularly in developing countries. Keeping in view the increased market demand of herbal drugs, it is essential to ensure their chemical quality prior to use. Raw drugs and herbs are usually collected from different places, which might be contaminated with various contaminants. It is pertinent to estimate the levels of heavy metals and other micronutrients, which could be affected by their presence in the surrounding environments. Heavy metals are known to pose a potential threat to terrestrial and aquatic biota. Keeping this in view, samples of ten plants or plant parts used in drug making were collected from local markets of Punjab for heavy metal and micronutrient estimation. It was found that the samples were contaminated having cadmium, lead, chromium, iron, manganese, and zinc. The highest mean level of cadmium (23.1 μg/kg) was found in Haritaki sample. Chromium concentration of the plant samples ranged between 7.25 and 1.34 μg/kg with the highest values was in Daruharidra and lowest in Pippali. The levels of these heavy metals were within permissible limits.

Invited for the cover of this issue is Ajay Venugopal from the Indian Institute of Science Education and Research Thiruvananthapuram. The cover image shows dication [TpMe2Bi]2+ catalyzing olefin hydrosilylation under mild conditions.

Web spam has the effect of polluting search engine results and decreasing the usefulness of search engines. Web spam can be classified according to the methods used to raise the web page's ranking by subverting web search engine's algorithms used to rank search results. The main types are: content spam, link spam and cloaking spam. There has been little or no work on automatically classifying web spam by type. This paper has two contributions, (i) we propose a Dual-Margin Multi-Class Hypersphere Support Vector Machine (DMMH-SVM) classifier approach to automatically classifying web spam by type, (ii) we introduce novel cloaking-based spam features which help our classifier model to achieve high precision and recall rate, thereby reducing the false positive rates. The effectiveness of the proposed model is justified analytically. Our experimental results demonstrated that DMMH-SVM outperforms existing algorithms with novel cloaking features.