Anomaly Detection on Credit Card Fraud no convergence

Part 1 (2019) Advanced (Part 1 v3)
1

Autoencoder Training Issues: Seeking Guidance

Hello everyone,

I’m currently working on an autoencoder project and have encountered some challenges with training the model. Despite trying various techniques and troubleshooting steps, I’ve been unable to achieve convergence. I’d appreciate any insights, suggestions, or advice the community might have.

Problem Description:

  • Model Type: Autoencoder
  • Data: [Credit Card Fraud, I normalized the data between 0 and 1]
  • Issue: The loss does not converge, and the model does not seem to be learning effectively.

Steps Taken:

  • Architectural Adjustments: [added more layers, changed activation functions, tried different bottlenecks]
  • Regularization Techniques: [L1/L2 regularization]
  • Optimization Algorithms: [RMSProp, Adam, SGD with momentum]
  • Learning Rate Scheduling: [different learning rates ]
  • Initialization Techniques: [Xavier/Glorot initialization]

Additional Information:

  • Loss Function [MSE Loss]

Autoencoder Model Definition

Below is the Python code defining an autoencoder model using PyTorch:

import torch
import torch.nn as nn

class Autoencoder(nn.Module):
    def __init__(self):
        super(Autoencoder, self).__init__()
        self.encoder = nn.Sequential(
            nn.Linear(29, 16),
            nn.ReLU(),
            nn.Linear(16, 12),  # Increase the bottleneck size
            nn.ReLU(),
            nn.Linear(12, 8),
            nn.ReLU()
        )
        self.decoder = nn.Sequential(
            nn.Linear(8, 12),   # Match the encoder's last layer
            nn.ReLU(),
            nn.Linear(12, 16),
            nn.ReLU(),
            nn.Linear(16, 29),
            nn.Sigmoid()
        )
        # Initialize weights using Xavier initialization
        for m in self.modules():
            if isinstance(m, nn.Linear):
                nn.init.xavier_uniform_(m.weight)

    def forward(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

# Create an instance of the Autoencoder model
model = Autoencoder()

# Print the model architecture
print(model)

I’ve consulted various resources and tried numerous approaches but haven’t been able to resolve the issue. If anyone has encountered similar challenges or has any suggestions for troubleshooting or improving the training process, I’d be grateful for your insights.

Open Google Colab Notebook

Thank you in advance for your time and assistance!