COCOGEN vs DAVINCI: A Human Evaluation of Structured Commonsense Graph Generation

Human evaluation confirms COCOGEN's advantage over DAVINCI in generating more relevant and accurate argument graphs. For EXPLAGRAPHS, COCOGEN consistently produces better semantic relations. In PROSCRIPT, both models have strengths, but COCOGEN generally excels in relevance and correctness.

Table of Links

Abstract and 1 Introduction

2 COCOGEN: Representing Commonsense structures with code and 2.1 Converting (T,G) into Python code

2.2 Few-shot prompting for generating G

3 Evaluation and 3.1 Experimental setup

3.2 Script generation: PROSCRIPT

3.3 Entity state tracking: PROPARA

3.4 Argument graph generation: EXPLAGRAPHS

4 Analysis

5 Related work

6 Conclusion, Acknowledgments, Limitations, and References

A Few-shot models size estimates

B Dynamic prompt Creation

C Human Evaluation

D Dataset statistics

E Sample outputs

F Prompts

G Designing Python class for a structured task

H Impact of Model size

I Variation in prompts

C Human ...