Using Code-LLMs for Symbolic and Structured Reasoning

This work builds on previous methods that flattened output graphs, innovating by using CodeLLMs to generate structured, Python-based outputs, which outperform traditional text-based models.

Table of Links

Abstract and 1 Introduction

2 COCOGEN: Representing Commonsense structures with code and 2.1 Converting (T,G) into Python code

2.2 Few-shot prompting for generating G

3 Evaluation and 3.1 Experimental setup

3.2 Script generation: PROSCRIPT

3.3 Entity state tracking: PROPARA

3.4 Argument graph generation: EXPLAGRAPHS

4 Analysis

5 Related work

6 Conclusion, Acknowledgments, Limitations, and References

A Few-shot models size estimates

B Dynamic prompt Creation

C Human Evaluation

D Dataset statistics

E Sample outputs

F Prompts

G Designing Python class for a structured task

H Impact of Model size

I Variation in prompts

5 Related work

Structured commonsense reasoning using LLMs Existing methods for structured commonsense generation typically flatten ...