• Mr. Radhe Lal Chadar
• Rajneesh Pachouri
• Mr. Anurag Jain

The exponential growth of networked systems has intensified cybersecurity threats, making real-time anomaly detection in network traffic a critical research imperative. This thesis proposes a novel framework for anomaly detection in network traffic leveraging advanced machine learning techniques including deep auto encoders, graph neural networks (GNNs), and transformer-based temporal modeling. Unlike traditional signature-based intrusion detection systems, the proposed approach employs unsupervised learning to establish baseline network behavior patterns from unlabeled traffic data, enabling the identification of zero-day attacks, DDoS floods, and subtle lateral movement activities. The methodology integrates three synergistic components: (1) a multi-scale convolutional auto encoder that captures spatial-temporal traffic patterns across packet, flow, and session levels; (2) a Graph Attention Network (GAT) that models dynamic communication topologies between network entities; and (3) a transformer architecture with positional encoding to capture long-range temporal dependencies in traffic sequences. Feature engineering incorporates 32 network flow metrics (IPFIX-compliant) including packet inter-arrival times, byte distributions, protocol entropy, and flow duration statistics. The reconstruction error from the auto encoder, combined with graph anomaly scores and temporal deviation metrics, feeds into a one-class SVM for final anomaly classification. Evaluation on benchmark datasets (CICIDS2017, UNSW-NB15, and MAWI real-world traces) demonstrates superior performance with 98.2% AUC, 4.1% false positive rate, and 12ms inference latency on 10Gbps traffic. The framework achieves 27% higher F1-score compared to Isolation Forest, 19% improvement over LSTM-based methods, and 3x faster inference than GAN-based approaches. Real-time deployment results on enterprise edge routers confirm scalability to 40Gbps throughput with sub-50ms detection latency. This research advances the state-of-the-art by addressing key limitations of existing methods: lack of multi-scale temporal modeling, inadequate handling of encrypted traffic, and poor generalization across attack families. The proposed solution offers a deployable, hardware-agnostic anomaly detection system suitable for 5G core networks, IoT edge gateways, and cloud security stacks, providing robust defense against evolving cyber threats through continuous adaptation and minimal human supervision.

Anomaly Detection, Network Traffic Analysis, Machine Learning, Deep Autoencoders, Graph Neural Networks, Transformers, Intrusion Detection Systems, Cybersecurity,

"Anomaly Detection in Network Traffic Using CatBoost, Extra Trees and Gradient Boosting Based Intrusion Detection System", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.13, Issue 1, page no.c492-c500, January-2026, Available :http://www.jetir.org/papers/JETIR2601259.pdf

"Anomaly Detection in Network Traffic Using CatBoost, Extra Trees and Gradient Boosting Based Intrusion Detection System", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.13, Issue 1, page no. ppc492-c500, January-2026, Available at : http://www.jetir.org/papers/JETIR2601259.pdf