CALL FOR PAPERS - COLING/ACL 2006 Conference Workshop
Task-Focused Summarization and Question Answering
http://research.microsoft.com/~lucyv/WS7.htm
Sydney, Australia
July 23, 2006
*** Submission Deadline: May 1, 2006 ***
Multilingual Summarization Evaluation
http://research.microsoft.com/~lucyv/MSE2006.htm
Workshop Description
This one-day workshop will focus on the challenges that the
Summarization and
QA communities face in developing useful systems and in developing
evaluation
measures. Our aim is to bring these two communities together to
discuss the
current challenges and to learn from each other's approaches,
following the
success of a similar workshop held at ACL-05, which brought together
the
Machine Translation and Summarization communities.
A previous summarization workshop (Text Summarization Branches Out,
ACL-04)
targeted the exploration of different scenarios for summarization,
such as
small mobile devices, legal texts, speech, dialog, email and other
genres.
We encourage a deeper analysis of these, and other, user scenarios,
focusing
on the utility of summarization and question answering for such
scenarios and
genres, including cross-lingual ones.
By focusing on the measurable benefits that summarization and question
answering has for users, we hope one of the outcomes of this workshop
will be
to better motivate research and focus areas for summarization and
question
answering, and to establish task-appropriate evaluation methods.
Given a user
scenario, it would ideally be possible to demonstrate that a given
evaluation
method predicts greater/lesser utility for users. We especially
encourage
papers describing intrinsic and extrinsic evaluation metrics in the
context of
these user scenarios.
Both summarization and QA have a long history of evaluations:
Summarization
since 1998 (SUMMAC) and QA since 1999 (TREC). The importance of
summarization
evaluation is evidenced by the many DUC workshops; in DUC-05,
extensive
discussions were held regarding the use of ROUGE, ROUGE-BE, and the
pyramid
method, a semantic-unit based approach, for evaluating summarization
systems.
The QA community has related evaluation issues for answers to complex
questions such as the TREC definition questions. Some common
considerations
in both communities include what constitutes a good answer/response to
an
information request, and how does one determine whether a "complex"
answer is
sufficient? In both communities, as well as in the distillation
component of
the 2005 DARPA program GALE, researchers are exploring how to capture
semantic
equivalence among components of different answers (nuggets, factoids
or SCUs).
There also have been efforts to design new automatic scoring measures,
such as
ROUGE-BE and POURPRE. We encourage papers discussing these and other
metrics
that report on how well the metric correlates with human judgments
and/or
predicts effectiveness in task-focused scenarios for summarization and
QA.
This workshop is a continuation of ACL 2005 for the summarization
community,
In which those interested in evaluation measures participated in a
joint
Workshop on evaluation for summarization and MT. As a sequel to the
ACL 2005
workshop, in which the results of the first Multilingual
multi-document
summarization evaluation (MSE) were presented
(http://www.isi.edu/~cyl/MTSE2005/MLSummEval.html),
we plan to report and discuss the results of the 2006 MSE evaluation.
In summary, we solicit papers on any or all of the following three
topics:
- Task-based user scenarios requiring question answering
(beyond factoids/lists) and/or summarization, across genres and
languages
- Extrinsic and intrinsic evaluations, correlating extrinsic measures
with
outcome of task completion and/or intrinsic measures with human
judgments
previously obtained.
- The 2006 Multilingual Multi-document Summarization Evaluation
Anyone with an interest in summarization, QA and/or evaluation is
encouraged
to participate in the workshop. We are looking for research papers in
the
aforementioned topics, as well as position papers that identify
limitations in
current approaches and describe promising future research directions.
SUMMARIZATION TASK: Multilingual Summarization Evaluation
Details for MSE 2006 will be available soon at
http://research.microsoft.com/~lucyv/MSE2006.htm.
For description and results of last year's MSE task, please see:
http://www.isi.edu/~cyl/MTSE2005.
Send email to lucy.vanderwende@microsoft.com to be added to MSE
mailing
list.
PAPER FORMAT:
Papers should be no more than 8 pages, formatted following the
guidelines that
will be made available on the conference Web site. The reviewing
process will
be blind, so authors' names, affiliations, and all self-references
should not
be included in the paper. Authors who cannot submit a PDF file
electronically
should contact the organizers at least one week prior to the May 1st
deadline.
Proceedings will be published in conjunction with the main HLT/NAACL
proceedings.
Details on how to submit your paper available on the website or by
contacting
the organizers.
IMPORTANT DATES:
Task-focused Summarization and Question Answering Workshop
Submission Due: May 1st
Notification of Acceptance: May 22nd
Camera-ready papers due: June 1st
Workshop date: July 23, 2006
Multilingual Summarization Evaluation:
Dates to be announced. Send email to lucy.vanderwende@microsoft.com
to be added to email distribution list.
WORKSHOP ORGANIZERS
Tat-Seng Chua, National University of Singapore;
chuats@comp.nus.edu.eg
Jade Goldstein, U.S. Department of Defense; jgstewa@afterlife.ncsc.mil
Simone Teufel, Cambridge University; simone.teufel@cl.cam.ac.uk
Lucy Vanderwende, Microsoft Research; lucy.vanderwende@microsoft.com
PROGRAM COMMITTEE
Regina Barzilay (MIT)
Sabine Bergler (Concordia University, Canada)
Silviu Cucerzan (Microsoft Research)
Hang Cui (National University of Singapore)
Krzysztof Czuba (Google)
Hal Daume III (USC/ISI)
Hans van Halteren (Radboud University Nijmegen, Netherlands)
Sanda Harabagiu (University of Texas, Dallas)
Chiori Hori (CMU)
Eduard Hovy (USC/ISI)
Hongyan Jing (IBM Research)
Guy Lapalme (University of Montreal)
Geunbae (Gary) Lee (Postech Univ, Korea)
Chin-Yew Lin (USC/ISI)
Inderjeet Mani (MITRE)
Marie-France Moens (Katholieke Universiteit Leuven, Belgium)
Ani Nenkova (Columbia University)
Manabu Okumura (Tokyo Institute of Technology)
John Prager (IBM Research)
Horacio Saggion (University of Sheffield, UK)
Judith Schlesinger (IDA/CCS)
Karen Sparck Jones (University of Cambridge)
Nicola Stokes (University of Melbourne)
Beth Sundheim (SPAWAR Systems Center)
Tomek Strzalkowski (University at Albany)
Ralph Weischedel (BBN)
Task-Focused Summarization and Question Answering
http://research.microsoft.com/~lucyv/WS7.htm
Sydney, Australia
July 23, 2006
*** Submission Deadline: May 1, 2006 ***
Multilingual Summarization Evaluation
http://research.microsoft.com/~lucyv/MSE2006.htm
Workshop Description
This one-day workshop will focus on the challenges that the
Summarization and
QA communities face in developing useful systems and in developing
evaluation
measures. Our aim is to bring these two communities together to
discuss the
current challenges and to learn from each other's approaches,
following the
success of a similar workshop held at ACL-05, which brought together
the
Machine Translation and Summarization communities.
A previous summarization workshop (Text Summarization Branches Out,
ACL-04)
targeted the exploration of different scenarios for summarization,
such as
small mobile devices, legal texts, speech, dialog, email and other
genres.
We encourage a deeper analysis of these, and other, user scenarios,
focusing
on the utility of summarization and question answering for such
scenarios and
genres, including cross-lingual ones.
By focusing on the measurable benefits that summarization and question
answering has for users, we hope one of the outcomes of this workshop
will be
to better motivate research and focus areas for summarization and
question
answering, and to establish task-appropriate evaluation methods.
Given a user
scenario, it would ideally be possible to demonstrate that a given
evaluation
method predicts greater/lesser utility for users. We especially
encourage
papers describing intrinsic and extrinsic evaluation metrics in the
context of
these user scenarios.
Both summarization and QA have a long history of evaluations:
Summarization
since 1998 (SUMMAC) and QA since 1999 (TREC). The importance of
summarization
evaluation is evidenced by the many DUC workshops; in DUC-05,
extensive
discussions were held regarding the use of ROUGE, ROUGE-BE, and the
pyramid
method, a semantic-unit based approach, for evaluating summarization
systems.
The QA community has related evaluation issues for answers to complex
questions such as the TREC definition questions. Some common
considerations
in both communities include what constitutes a good answer/response to
an
information request, and how does one determine whether a "complex"
answer is
sufficient? In both communities, as well as in the distillation
component of
the 2005 DARPA program GALE, researchers are exploring how to capture
semantic
equivalence among components of different answers (nuggets, factoids
or SCUs).
There also have been efforts to design new automatic scoring measures,
such as
ROUGE-BE and POURPRE. We encourage papers discussing these and other
metrics
that report on how well the metric correlates with human judgments
and/or
predicts effectiveness in task-focused scenarios for summarization and
QA.
This workshop is a continuation of ACL 2005 for the summarization
community,
In which those interested in evaluation measures participated in a
joint
Workshop on evaluation for summarization and MT. As a sequel to the
ACL 2005
workshop, in which the results of the first Multilingual
multi-document
summarization evaluation (MSE) were presented
(http://www.isi.edu/~cyl/MTSE2005/MLSummEval.html),
we plan to report and discuss the results of the 2006 MSE evaluation.
In summary, we solicit papers on any or all of the following three
topics:
- Task-based user scenarios requiring question answering
(beyond factoids/lists) and/or summarization, across genres and
languages
- Extrinsic and intrinsic evaluations, correlating extrinsic measures
with
outcome of task completion and/or intrinsic measures with human
judgments
previously obtained.
- The 2006 Multilingual Multi-document Summarization Evaluation
Anyone with an interest in summarization, QA and/or evaluation is
encouraged
to participate in the workshop. We are looking for research papers in
the
aforementioned topics, as well as position papers that identify
limitations in
current approaches and describe promising future research directions.
SUMMARIZATION TASK: Multilingual Summarization Evaluation
Details for MSE 2006 will be available soon at
http://research.microsoft.com/~lucyv/MSE2006.htm.
For description and results of last year's MSE task, please see:
http://www.isi.edu/~cyl/MTSE2005.
Send email to lucy.vanderwende@microsoft.com to be added to MSE
mailing
list.
PAPER FORMAT:
Papers should be no more than 8 pages, formatted following the
guidelines that
will be made available on the conference Web site. The reviewing
process will
be blind, so authors' names, affiliations, and all self-references
should not
be included in the paper. Authors who cannot submit a PDF file
electronically
should contact the organizers at least one week prior to the May 1st
deadline.
Proceedings will be published in conjunction with the main HLT/NAACL
proceedings.
Details on how to submit your paper available on the website or by
contacting
the organizers.
IMPORTANT DATES:
Task-focused Summarization and Question Answering Workshop
Submission Due: May 1st
Notification of Acceptance: May 22nd
Camera-ready papers due: June 1st
Workshop date: July 23, 2006
Multilingual Summarization Evaluation:
Dates to be announced. Send email to lucy.vanderwende@microsoft.com
to be added to email distribution list.
WORKSHOP ORGANIZERS
Tat-Seng Chua, National University of Singapore;
chuats@comp.nus.edu.eg
Jade Goldstein, U.S. Department of Defense; jgstewa@afterlife.ncsc.mil
Simone Teufel, Cambridge University; simone.teufel@cl.cam.ac.uk
Lucy Vanderwende, Microsoft Research; lucy.vanderwende@microsoft.com
PROGRAM COMMITTEE
Regina Barzilay (MIT)
Sabine Bergler (Concordia University, Canada)
Silviu Cucerzan (Microsoft Research)
Hang Cui (National University of Singapore)
Krzysztof Czuba (Google)
Hal Daume III (USC/ISI)
Hans van Halteren (Radboud University Nijmegen, Netherlands)
Sanda Harabagiu (University of Texas, Dallas)
Chiori Hori (CMU)
Eduard Hovy (USC/ISI)
Hongyan Jing (IBM Research)
Guy Lapalme (University of Montreal)
Geunbae (Gary) Lee (Postech Univ, Korea)
Chin-Yew Lin (USC/ISI)
Inderjeet Mani (MITRE)
Marie-France Moens (Katholieke Universiteit Leuven, Belgium)
Ani Nenkova (Columbia University)
Manabu Okumura (Tokyo Institute of Technology)
John Prager (IBM Research)
Horacio Saggion (University of Sheffield, UK)
Judith Schlesinger (IDA/CCS)
Karen Sparck Jones (University of Cambridge)
Nicola Stokes (University of Melbourne)
Beth Sundheim (SPAWAR Systems Center)
Tomek Strzalkowski (University at Albany)
Ralph Weischedel (BBN)