> For the complete documentation index, see [llms.txt](https://alfredo-reyes-montero.gitbook.io/openai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://alfredo-reyes-montero.gitbook.io/openai/application/generate-custom-text-content.md).

# Generate Custom Text Content

We will be exploring the steps required to retrain GPT-2 (117M) using custom text dataset. For start, GPT-2 is the advanced version of a transformer-based model that was trained to generates synthetic text samples from a variety of user-prompts as input. 

The original version has 1.5GB parameters but the creator, OpenAI team did not released the pre-trained model due to their concerns about malicious applications of the technology. Having said that, they did released a smaller version which has 117MB parameters that can be retrained on custom text dataset.

![](https://4213617597-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LqMg4qkOis4QLHkiBUK%2F-LqfdRLhdopz6nyaoLL5%2F-LqfdYhG0mRzea6-1vWS%2Fimage.png?alt=media\&token=268aaa43-fe4f-4025-b96f-eaa8764cb124)