Conclusion and Future work
This thesis presented a rule based automatic question generation system that
focuses on both question generations from sentences and paragraphs. Especially,
with respect to METEOR metric, the designed system significantly outperforms all
other systems in automatic evaluation stage. Banerjee et al. (2005) demonstrated that
METEOR has significantly enhanced correlation with human evaluators. So, our
results confirm that statement by performing human evaluation study. In conclusion,
the designed system significantly outperforms all other systems in human evaluation
study by generating the most natural (human-like) questions.
For deciding
between who and what questions, we proposed solution. This
problem is one of the lexical challenges that we have stated in. Our results in Table
shows that, with 4.262 correctness score, we correctly differentiate between who and
what questions. Also, for another lexical challenge, non-compositionality that is
stated, we proposed solution. Our predefined dictionary does not cover all idioms.
Also, some types of idioms cannot be covered with predefined dictionary. This issue
will be explored in the future work.
Currently, our templates do not achieve the
best performance across all
question categories. If we look at Table, S-V-number and S-V-ARGM-MNR (how)
type of questions has a low correctness score. In addition, in order to improve the
performance of paragraph based questions in all templates, we need to investigate
how to better use the paragraph-level information. This
is one of the discourse
challenges that we have mentioned. Information conveyed from one sentence to
other is a problematic issue. So, we leave this issue to future work. Finally, some
templates fit better with some topics than others. For instance, S-V-attr and S-V-
oprd templates that is stated, works better with noun phrases that are suitable with
descriptive questions. For definition questions, other techniques need to be explored
in the future work.
Also, adapting the designed system to Uzbek language would not be easy due
to lack of syntactic and semantic parsers. Without
high-performance parsers,
adapting predefined rules into Uzbek language would not give a similar
performance.
Dostları ilə paylaş: