A classifier can be weak for a number of reasons, and it mainly depends on characteristics of the data. For example, if the data are not linearly separable, then linear regression will be weak (poor correlation between predicted class and true class labels). However, if the data are linearly separable, then other classifiers may not work as well as linear regression. If you used ensemble methods (committee of classifiers), you could compare results across classifiers. You didn't mention anything about boosting, which is fundamental to weak learner applications. Without going into detail, you could start with the most basic classifiers such as k-nearest neighbors(kNN), Naive Bayes classifier(NBC), learning vector quantization(LVQ), linear discriminant analysis(LDA), and then linear regression. If the data depart from normality, then LDA may breakdown since covariance matrices are used, and if nonlinearly-separable data are present, regression may have larger error. Certainly, as you increase classifier complexity such as with support vector machines, random forests, artificial neural networks, the earlier mentioned classifiers (kNN, NBC, LDA, LREG) may not work as well. The main issue is that, as soon as you mention "weakness," the theory of boosting overarches everything you are doing, and boosting involves more complex methods for making a weak learner stronger. Hence, to pursue what you want to do, you may get "trapped" in boosting theory and become forced to only consider boosting issues -- which has its own unique assumptions. Make sure you really have to work with a weak classifier instead of comparing classifiers via an ensemble. (See Kuncheva's papers on the diversity issue for an ensemble of classifiers).