Liang, S: An Empirical Study of Automatic Summarisation

This research aimed to improve search engine result summaries to help internet users make quick and accurate relevance judgements. Term Order and Query Term Order(QTO) algorithms were developed in order to produce better search engine result summaries. Six sentence weighting schemes were constructed in different weighting components combinations with the aim of comparing the effectiveness between QTO and Query Term Frequency (QTF). The literature on automatic summarisation evaluation is classified into intrinsically motivated Gold Standard/Baseline and Subjective Scoring evaluations, and extrinsically motivated Task Based evaluation. In order to triangulate evidence of the usefulness of the QTO algorithm, Document Understanding Conference data was used for intrinsic evaluation and online English web data was used for both intrinsic and extrinsic evaluations. The QTO summarisation system was compared against that of Google. Representativeness, Judgeability and Scanning-Speed were the three tasks in the evaluations, and the summary quality was derived from the measurements of the three tasks.