"Simulated molecular evolution" or computer-generated artifacts?

Abstract

1. The authors define a function with value 1 for the positive examples and 0 for the negative ones. They fit a continuous function but do not deal at all with the error margin of the fit, which is almost as large as the function values they compute. 2. The term "quality" for the value of the fitted function gives the impression that some biological significance is associated with values of the fitted function strictly between 0 and 1, but there is no justification for this kind of interpretation and finding the point where the fit achieves its maximum does not make sense. 3. By neglecting the error margin the authors try to optimize the fitted function using differences in the second, third, fourth, and even fifth decimal place which have no statistical significance. 4. Even if such a fit could profit from more data points, the authors should first prove that the region of interest has some kind of smoothness, that is, that a continuous fit makes any sense at all. 5. "Simulated molecular evolution" is a misnomer. We are dealing here with random search. Since the margin of error is so large, the fitted function does not provide statistically significant information about the points in search space where strings with cleavage sites could be found. This implies that the method is a highly unreliable stochastic search in the space of strings, even if the neural network is capable of learning some simple correlations. 6. Classical statistical methods are for these kind of problems with so few data points clearly superior to the neural networks used as a "black box" by the authors, which in the way they are structured provide a model with an error margin as large as the numbers being computed.7. And finally, even if someone would provide us with a function which separates strings with cleavage sites from strings without them perfectly, so-called simulated molecular evolution would not be better than random selection.Since a perfect fit would only produce exactly ones or zeros,starting a search in a region of space where all strings in the neighborhood get the value zero would not provide any kind of directional information for new iterations. We would just skip from one point to the other in a typical random walk manner.

A Brief History of Protein Sorting Prediction

“…The Schneider and Wrede 1994 paper [ 54 ] was harshly criticized in a comment by Darius and Rojas [ 56 ] who, among other points, wrote: “The term “quality” for the value of the fitted function gives the impression that some biological significance is associated with values of the fitted function strictly between 0 and 1, but there is no justification for this kind of interpretation and finding the point where the fit achieves its maximum does not make sense.”…”

pkaPS: prediction of protein kinase A phosphorylation sites with the simplified kinase-substrate binding model

“…In such circumstances, the usage of machine learning software packages as "black boxes" for autonomous extraction of score function parameters without human interference and without explicitly considering the physico-chemical and biological realities of the problem under study can become dangerous. In their letter to the editors of the Biophysics Journal in 1994 [51], Frank Darius and Raul Rojas analyze the difficulties arising from the discrepancy between the very high dimension of the parameter space in modern machine learning approaches and scarce data when exemplarily criticizing an alternative signalpeptide predictor. To summarize, the problem is that the calculated parameters are not reliable and it is not clear whether the correlations found are numerical noise of the data or biologically meaningful.…”

Topology Preserving Neural Networks for Peptide Design in Drug Discovery

“…This is done by training several (10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20) TPNNs on randomly chosen subsets of the training data where the training starts with random weight initializations. To compute the output of the ensemble for one input sequence, the output variables of all TPNNs belonging to the ensemble are averaged.…”

“…An overview could be found in Weekes and Fogel [11] or in Terfloth and Gasteiger [12] Earlier works from Schneider and Wrede [13] proposed the use of neural networks and computerbased evolutionary search for peptide design. Their work was criticized by Darius and Rojas [14] who questioned the statistical significance of their results in general. But Schneider et al demonstrated later that their method can generate novel peptides with substantial biological activity in real world experiments [15].…”

“…An overview could be found in Weekes and Fogel [11] or in Terfloth and Gasteiger [12] Earlier works from Schneider and Wrede [13] proposed the use of neural networks and computerbased evolutionary search for peptide design. Their work was criticized by Darius and Rojas [14] …”