About

What is KnowLM

The KnowLM project
KnowLM is an open-source language model framework for knowledge enhancement, designed to assist researchers and developers in better managing updates and inaccuracies in large models by providing a flexible and customizable toolkit and releasing corresponding models.

The technology, capabilities and content of KnowLM are as follows:

  • Knowledge Prompting
  • Knowledge Editing
  • Knowledge Interaction
Learn More

Knowledge Prompting.

KnowPrompt.

KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction.  WWW2022.

"Knowledge-informed Prompt Learning" seeks to enhance pre-trained models' understanding of specific domains and tasks by incorporating domain-related knowledge, such as knowledge graphs and text examples, through "Knowledge as Prompt".

This approach aims to unlock the full potential of pre-trained models and improve their performance significantly.

For More

Knowledge Editing.

EasyEdit.

Editing Large Language Models: Problems, Methods, and Opportunities. EMNLP2023.

Model Editing, aims to adjust an initial base model's behavior on the particular edit descriptor efficiently without influencing the model behavior on other samples.

Model Editing has strong connections with following topics:updating and fixing bugs for large language models, locating knowledge in large language models and lifelong learning, unlearning.

For More

Knowledge Interaction.

KnowAgent.

KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents.

Knowledge Interaction refers to the dynamic knowledge exchange and feedback between models and humans, environment, tools, or other models.

It is an important process to realize complex planning, agent intelligence, and large language model embodied cognition.

For More

Technologies for training LLMs

The full-scale pre-training code (providing conversion, construction, and loading of large corpora) and LoRA instruction fine-tuning code are open-sourced (support multi-machine multi-GPU). Provide examples for knowledge prompt, editing and interaction (still developing).

Github

Resources

KnowLM Zoo

KnowLM Model Zoo

  • We have currently open-sourced the base model of ZhiXi-13B and the extraction model of ZhiXi-13B, including the upcoming release of the base model KnowLM-7B based on LLaMA2.

Portfolio

Supported LLMs

Related LLMs that using the technologies from KnowLM.

  • All

OneKE

OceanGPT

PolicyQA

Portfolio

What we've done

Paper Lists

Related works regarding knowledge prompt, editing and interaction.

  • All

KnowPrompt

Knowledge-aware prompt-tuning approach with synergistic optimization for relation extraction.

EasyEdit

An easy-to-use framework to edit LLMs including GPT-J, Llama1/2, GPT-NEO, GPT2, T5.

KnowAgent

Knowledge-Augmented Planning for LLM-Based Agents.

RetroPrompt

Constructs an open-book knowledge-store from training instances and implements a retrieval mechanism for prompt learning.

RAP

A model-agnostic approach that utilizes schema and knowledge from human-annotated and weakly-supervised data as prompts.

TRICE

A two stage end-to-end framework that enables the model to continually learn through feedback derived from tool execution.

Acknowledgments

Our Sincere Thanks

Our sincerest thanks are extended to Langchain, Transformers, FastEdit, ROME, GRACE, MELO , PMET , and Self-Instruct for their invaluable contributions to our project. We have integrated parts of their source code into our work, and for this, we are deeply appreciative.

Furthermore, we are grateful for the ongoing support and collaboration from our community. Special recognition goes to those who have diligently reported issues and shared their technical expertise. Your collective efforts have been instrumental in our project's success. 🙌