2
$\begingroup$

I am working on Anomaly detection model problem for a finance data set - set of gift card activation transactions. My team member suggested an idea that " First train the model with normal instances of data and to the trained model pass the anomalous data during testing to see whether the model will be able to detect the anomaly or not." Is it possible like this with any Machine Learning algorithms. If so, can anybody suggest what algorithms can be used for this idea?

An additional question: How about the situation when there is no properly labelled fraudulent data(as in my case) and we have just transactional information(merchant,store,card amount, physical address of store, type of card, brand of gift card and timestamp variables) and not any info related to customers who bought the cards. Does semi-supervised learning works? I do not have experience in this type of ML approach. Can anybody suggest good resources for this and also suggest can Semi-supervised learning or hybrid supervised learning works for anomaly/ fraud detection?

|improve this question
$\endgroup$
  • 2
    $\begingroup$ I'm voting to close this question as off-topic because it's an interesting topic but it fits better on the DataScience stack. However, it's also quite broad. $\endgroup$ – Bob Jansen♦ Sep 7 at 7:05
  • $\begingroup$ As for the first question, I would recommend the classic ensemble of Logit, k-nearest neighbors, decision tree/random forest and possibly linear discriminant analysis. You can stack a SVM and a neural network, too, but the best way is to start from very simple deeply cross-validated ensembles and then slowly add models. As for the second question, you have no alternatives to clustering: usually t-SNE > DBSCAN > k-means, but try all of them. $\endgroup$ – Lisa Ann Sep 7 at 7:11
  • $\begingroup$ @LisaAnn Thank you! I'll definitely try the model you have said. Can we do ensembles on unsupervised learning models? Actually, I am novice to ensembles. If you could specify an links with practical example of ensemble it would be great. $\endgroup$ – tjrdata Sep 9 at 20:38
  • $\begingroup$ Ensemble learning is a combination (in the broadest sense) of predictions aimed at reducing variance without increasing overfitting. Take some simple classification models: the simplest ensemble is to average the predictions. You can play with caretEnsemble for more complicated stacking schemes. Unfortunately, I don't think that you can do the same with clustering in a consistent way. $\endgroup$ – Lisa Ann Sep 10 at 8:59