What is fine tuning? Differences from transfer learning and RAG, creation steps, application examples, and points to note!

Fine-tuning is a method for re-learning a pre-trained AI model according to a target task. In particular, fine-tuning is attracting wide attention as a method for optimizing LLMs (Large Language Models) for specific business challenges and industry-specific needs. However, many questions remain regarding what specific steps should be followed and what to watch out for, and many companies hesitate to introduce it.

In this article, we will introduce the basic overview of fine-tuning while considering the differences from transfer learning and RAG (Retrieval-Augmented Generation). We will also explain in detail the data creation procedures for practice, application examples, and points to note during implementation. The content allows you to understand effective ways to use fine-tuning and can be helpful when actually introducing it to a business.

1. What is Fine-tuning?

Fine-tuning is the process of re-training an AI model that has already been trained on massive amounts of data with additional data to fit a specific purpose or environment.

For example, by providing additional learning to a general-purpose LLM (Large Language Model) that has learned natural language data from multiple fields with data from a specific field such as medicine or the latest data, the parameter constants are changed to develop a new medical-specialized LLM. If likened to sports, it is like taking a rookie player (pre-trained model) who already has a certain level of basic physical strength and motor skills and subjecting them to additional practice (fine-tuning) to teach them specific sports movements, tool usage, strategies, and rules to turn them into an immediate asset.

In such cases, with traditional machine learning models, it was basically necessary to learn from scratch, requiring enormous data and computational resources. However, as fine-tuning methods have become common, it has become possible to output according to specific uses while significantly reducing learning costs based on existing high-performance models. Since the introduction of AI technology can be performed more efficiently by utilizing fine-tuning, it is adopted in various fields.

Difference Between Fine-tuning and Transfer Learning

Transfer learning refers to a broad concept of repurposing knowledge learned in one area for other tasks. Fine-tuning is one of the specific methods for realizing transfer learning. Fine-tuning is characterized by adjusting and optimizing internal model parameters according to the characteristics of a new task while utilizing general-purpose knowledge obtained in pre-training. As an example of a method other than fine-tuning for realizing transfer learning, there is also the method of "using a pre-trained model as a feature extractor."

Difference Between Fine-tuning and RAG

Fine-tuning and RAG are both methods aimed at improving AI model performance to adapt to new tasks, but they have different mechanisms. The following are the main differences between fine-tuning and RAG.

Fine-tuning is the process of optimizing the model itself for a specific task, while RAG is a method for supplementing the generative model by utilizing external information. The overview of RAG is explained in detail in "What is RAG? Explaining its structure, functions, installation procedures, use cases, and points to note!", so please take a look at it as well.

Fine-tuning Methods

In fine-tuning, a wide variety of methods have been developed. The following are the main methods.

In recent years, research on Parameter-Efficient Fine-tuning (PEFT), which efficiently adapts models while keeping computational resources and memory usage low, has been active. Various methods such as the following have been proposed:

• Prompt Tuning:
  Adds a learnable virtual token to the beginning of the input (prompt) and learns only the parameters of that virtual token.
• Adapter Layers:
  Inserts small new layers (adapters) into the existing network and learns only those adapters.
• LoRA (Low-Rank Adaptation):
  Adds low-rank matrices to specific layers within the model and learns only the parameters of those low-rank matrices.
• (IA)^3:
  Introduces and trains learnable vectors to scale (increase/decrease) the output values of internal activation functions.

Among the above, LoRA is widely used because it can demonstrate high performance with relatively few parameters. Also, (IA)^3 may suffice with very few parameters. Which method to select depends on the target task, available computational resources, and the required performance level.

2. 4 Benefits of Performing Fine-tuning

Fine-tuning contributes significantly to efficient AI development and accuracy improvement. Here, we introduce four benefits obtained by performing fine-tuning.

Speeding up AI Development

In fine-tuning, by utilizing large pre-trained models, there is no need to learn from scratch. Therefore, it is possible to build practical models in a short period. Since it is a mechanism that repurposes the knowledge of base models that already have extensive knowledge, it can significantly reduce the amount of computation and time required for advanced learning. Utilizing fine-tuning allows for shortening the lead time to market for products and services.

Adaptation to Specialized Fields

Through fine-tuning, adaptation and customization for specialized tasks in various industries become easy. In particular, by utilizing fine-tuning with generative AI, outputs that reflect industry or domain-specific terminology, special expressions, and unique writing styles can be realized. Fine-tuning makes it possible to optimize an AI model as your own "company-specific" AI.

Reduction of Development Costs

Since the base model already has extensive knowledge, the additional learning required for fine-tuning is effective with a relatively small amount of data. Therefore, the costs of collecting data, purchasing datasets, and performing annotation can be kept down. Also, computational resources (such as high-performance GPUs) can be significantly reduced compared to learning a model from scratch. Thus, even companies with limited human resources, such as startups, can efficiently perform AI development.

High Efficiency with High Performance from Small Data

The strength of fine-tuning is that high-quality output can be realized even with a small amount of data. This is because models that have already learned general and universal knowledge can be utilized. Utilizing high-performance base models developed by IT giants like Google, OpenAI, and Meta is a major benefit. Through fine-tuning, you can incorporate the benefits of state-of-the-art AI technology into your business. High output accuracy can be maintained while short-term model adaptation is possible even from limited data.

3. Business Application Examples of Fine-tuning

In recent years, fine-tuning has been applied in various fields. Here, we introduce application examples that are attracting particular attention.

LLM (Large Language Model)

By fine-tuning highly versatile LLMs like the GPT used in OpenAI's ChatGPT, you can reflect knowledge and terminology specialized for specific industries or tasks while obtaining the flexible and high-precision answers unique to ChatGPT. For example, it is possible to obtain high-precision generation results required for specialized tasks such as the following:

• Marketing content utilized in SNS posts, industry reports, etc.
• Various texts constituting corporate branding
• Customer support responses

By applying fine-tuning to LLMs, more personalized and advanced outputs can be obtained. The benefits and drawbacks, enterprise introduction processes, and use cases of LLMs are explained in detail in "What is an LLM? Explaining the system, types, benefits, implementation procedures, and use cases!", so please take a look at it as well.

Image Recognition

In the field of image recognition, fine-tuning is performed to match specific types of images such as medical images, satellite images, and industrial images based on large-scale pre-trained models like ImageNet. For example, detection accuracy for lesions and abnormalities in medical images improves, and in industrial applications, anomaly detection on production lines can be performed with good precision. Even with limited data, the precision and efficiency of AI-based image recognition improve significantly, realizing rapid and high-precision decision support in various industries. The types, use cases, model construction methods, and issues of image recognition are explained in detail in "What is image recognition? Types, mechanisms, AI development processes, case studies and key considerations.", so please take a look at it as well.

4. Common Procedures for Fine-tuning

Here are the standard procedures for fine-tuning.

1. Selection of a Pre-trained Model

Choose a pre-trained model suitable for the task you want to solve and the available computational resources (GPU specs, budget, etc.). For example, candidates include BERT, GPT, and T5 for natural language processing, and ResNet or VGG for image recognition.

2. Data Collection

To make fine-tuning successful, collecting data related to the task is particularly important. The quality, quantity, and diversity of data significantly influence the model's performance. When collecting data, utilize existing datasets, public data, or data held internally. If necessary, it is also recommended to supplement data using web scraping, external APIs, or crowdsourcing. 6 points to note during data collection are explained in detail in "Things to keep in mind when requesting annotation data collection", so please take a look at it as well.

3. Execution of Annotation and Data Preprocessing

Perform accurate labeling and annotation on the collected data. In particular, data annotation is a vital task in AI model learning. The precision of annotation directly links to the precision of the model. The types of data targeted for annotation, reasons why annotation is important, and the annotation process are explained in detail in "What is annotation? Why is it necessary for AI use? Explaining the process and work involved", so please take a look at it as well.

To format the data for learning, preprocessing is also necessary according to the data type. For example, for image data, this includes resizing, noise reduction, and normalization; for text data, it includes tokenization and stopword removal.

4. Selection of Fine-tuning Method and Model Adjustment

The procedures from here vary depending on the AI development platform used, such as Amazon SageMaker. First, determine the fine-tuning method considering computational resources, required precision, and the amount of knowledge to be updated. Then, train and adjust the model in a way suitable for that platform, setting parameters such as the following:

• Learning Rate
• Batch Size
• Epochs (how many times the dataset is learned)

Since these greatly influence the success of fine-tuning, set and adjust them carefully.

5. Execution and Evaluation of Fine-tuning

Start model training (additional learning) using the prepared dataset and set hyperparameters. Finally, evaluate the fine-tuned AI model using a validation dataset or test dataset. Confirm performance using evaluation metrics appropriate for the task, such as F1 score or various loss functions.

5. Points to Note When Performing Fine-tuning

To utilize fine-tuning effectively, there are several points to note.

Quality Depends on Additional Learning Data

The results of fine-tuning depend heavily on the quality of the additional learning dataset used. Inaccurate labels, noise-filled data, and biased data can cause model performance degradation or unexpected behavior. Also, if the amount of data is too small, the model may not be able to learn the task sufficiently. Therefore, data annotation is essential for success when performing fine-tuning.

Catastrophic Forgetting

In full fine-tuning, "Catastrophic Forgetting" may occur, where the model "forgets" the general-purpose knowledge acquired during pre-training while learning a new task. Catastrophic forgetting can significantly degrade performance in tasks other than the fine-tuned one. Proper setting of the learning rate and consideration of Parameter-Efficient Fine-tuning are required.

Overfitting

Overfitting is a phenomenon where the model fits the prepared fine-tuning dataset too excessively, losing its generalization performance (adaptability) for unknown data. Measures such as preparing validation data separately from training data, ending learning at an appropriate timing while monitoring validation performance, and setting appropriate hyperparameters (especially learning rate and epochs) are necessary.

Securing Computational Resources

Fine-tuning can consume large amounts of computational resources. Especially when using large pre-trained models, an environment with sufficient GPU, TPU, and memory is required. Estimate computational resources in advance and prepare appropriate hardware to avoid resource shortages or excessive allocation.

6. Summary

By utilizing fine-tuning in LLM and image recognition fields, high-precision output and recognition become possible with small amounts of data, leading to shorter AI development lead times. This is because utilizing pre-trained models input with general data allows for efficiently adapting models to specific tasks even with limited data, yielding high-precision results. However, if data annotation work is of low quality, bias and noise will have an impact, potentially reducing versatility or increasing the risk of overfitting. Therefore, accurate annotation is indispensable for the success of fine-tuning.

Author

Latest Articles