Work

FLAMES : Fine-tuned Large Language Model for Invariant Synthesis

Year: 2024
Type: dissertation
Abstract: Smart contracts face significant security challenges due to their immutable nature and high-stakes applications. Traditional methods for securing these contracts, such as manual audits and static analysis, often fall short in detecting and repairing subtle vulnerabilities. This thesis presents a novel approach to improving smart contract security using large language models (LLMs) that are fine-tuned for invariant generation. We have developed a framework called FLAMES that utilizes fine-tuned CodeLlama models to automatically generate invariants for Solidity smart contracts, thereby improving security and functionality. Our approach incorporates a Fill-in-the-Middle (FIM) token infilling technique, enabling the model to predict missing security checks with high contextual relevance. To support this, we constructed the DISL dataset, the largest dataset in the world and optimized for Large Language Models training. We demonstrate that our model generates meaningful security invariants through rigorous testing, significantly enhancing baseline performance. This work provides empirical insights into using LLMs for automated security in smart contracts and addresses critical gaps in Automatic Program Repair for Solidity.
Author: Morello, Gabriele
Language: English
Open Access status: green
OpenAlex ID: W7019278914
Cites: 28
Cited by:
Related to: 10