Thank you for agreeing to review for CVPR 2016. To maintain a
high-quality technical program, we rely very much on the time and
expertise of our reviewers. This document explains what is expected
of all members of the Reviewing Committee for CVPR 2016.
|• November 6:
||Paper submission deadline
|• November 16-17:
||paper bidding stage
|• November 27:
||Papers assigned to reviewers
|• January 8:
|• January 18:
||Reviews released to authors
|• January 18-25:
|• Jan 26 - Feb 12:
||Paper discussion with Area Chairs and fellow reviewers
|• February 12:
||Final reviewer recommendation due
|• March 2:
||Final decision to authors
Authors were asked to take reasonable efforts to hide their
identities, including not listing their names or affiliations and
omitting acknowledgments. This information will of course be included
in the published version. Reviewers should also make all efforts to
keep their identity invisible to the authors. Please see
the Author Guidelines for details
on how authors have been instructed to preserve anonymity, including
guidelines for referencing one's own prior work. We realize that with the increase in popularity of publishing technical reports and ArXiv papers, sometimes the authors of a paper may be known to the reviewer. As voted on at the CVPR 2015 PAMI-TC meeting, ArXiv papers are not considered prior work since they have not been peer reviewed.
Therefore, you should review your CVPR papers independently as if the ArXiv papers didn't exist. Citations to these papers are not required and failing to cite or beat performance of arXiv papers are not grounds for rejection. Reviewers should make every effort to treat papers fairly whether or not they know (or suspect) who wrote them.
New to CVPR 2016, we have provided some specific examples in the Reviewer FAQs below.
As soon as you get your reviewing assignment, please go through all
the papers to make sure that (a) there is no obvious conflict with you
(e.g., a paper authored by your recent collaborator from a different
institution) and (b) you feel comfortable to review the paper
assigned. If either of these issues arise, please let us know right
emailing the Program Chairs.
Look for what's good or stimulating in the paper. Minor flaws can
be corrected and shouldn't be a reason to reject a paper. CVPR as a
conference is looking for new ideas. We recommend that you embrace
novel, brave concepts, even if they have not been tested on many
datasets. For example, the fact that a proposed method does not
exceed the state of the art accuracy on an existing benchmark dataset
is not grounds for rejection by itself. Acceptance and rejection
decisions should not be determined solely by the method's raw
performance. Rather, it is important to weigh both the novelty and
potential impact of the work alongside the reported performance. Each
paper that is accepted should be technically sound and make a
contribution to the field.
Please be specific and detailed in your reviews. In the discussion
of related work and references, simply saying "this is well known" or
"this has been common practice in the industry for years" is not
sufficient: cite specific publications, including books, or public
disclosures of techniques.
Your main critique of the paper should be written in terms of a
list of strengths and weaknesses of the paper. Use bullet points
here, and explain your arguments. Your discussion, sometimes more than
your score, will help the authors, fellow reviewers, and Area Chairs
understand the basis of your opinions, so please be thorough. Your
reviews will be returned to the authors, so you should include
specific feedback on ways the authors can improve their papers. For
more suggestions on writing your reviews, read the section below on
Writing Technical Reviews.
When you have finished with your review, you should destroy any
paper manuscript and/or supporting material you received. See the
Ethics guidelines below.
Here are some recommendations that may help you as you do this very
We volunteer our time by reviewing papers that are written by other
researchers in our field. We recommend that you approach your reviews
in this spirit of volunteerism. Your reviews make you a gatekeeper in
helping decide which papers are ready for publication. Just as
important, however, is to provide feedback to the authors so that they
may improve their work. Try to write your review in a way that the
authors can benefit from. We suggest reading a paper and then thinking
about it over the course of several days before you write your
The tone of your review is also important. A harshly written review
will be disregarded by the authors, regardless of whether your
criticisms are true. If you take care, it is always possible to word
your review diplomatically while staying true to your thoughts about
the paper. Put yourself in the mindset of writing to someone you wish
to help, such as a respected colleague who wants your opinion on a
concept or a project.
Here are some specific issues to keep in mind as you write your reviews:
- Short reviews are unhelpful to authors, other reviewers, and Area
Chairs. If you have agreed to review a paper, you should take enough
time to write a thoughtful and detailed review.
- Be specific when you suggest that the writing needs to be
improved. If there is a particular section that is unclear, point it
out and give suggestions for how it can be clarified.
- Don't give away your identity by asking the authors to cite
several of your own papers.
- Be specific about novelty. Claims in a review that the submitted
work "has been done before" MUST be backed up with specific references
and an explanation of how closely they are related. At the same time,
for a positive review, be sure to summarize what novel aspects are
most interesting in the strengths.
- Citations to papers that have only been published without review (e.g. ArXiv or Technical reports) are not required. Therefore, missing these citations is not grounds for rejecting a paper.
- If you think the paper is out of scope for CVPR's subject areas,
clearly explain why in the review. You may find the Call for Papers
Then suggest other publication possibilities (journals, conferences,
workshops) that would be a better match for the paper.
- Avoid referring to the authors by using the phrase "you". These
phrases should be replaced by "the authors" or "the paper." Referring
to the authors as "you" can be perceived as being confrontational,
even though you do not mean it this way.
Be generous about giving the authors new ideas for how they can
improve their work. Your suggestions may be very specific (for
example, "this numerical solver would be better for your application")
or may be more general in nature. You might suggest a new dataset that
could be tried, or a new application area that might benefit from
their tool. You may tell them how their idea can be generalized beyond
what they have already considered. A thoughtful review not only
benefits the authors, but may benefit you as well. Remember that your
reviews are read by other reviewers and especially the Area Chairs, in
addition to the authors. Being a helpful reviewer will generate good
will towards you in the research community.
As a reviewer for CVPR, you have the responsibility to protect the
confidentiality of the ideas represented in the papers you
review. CVPR submissions are not published documents. The work is
considered new or proprietary by the authors; otherwise they would not
have submitted it. Of course, their intent is to ultimately publish to
the world, but most of the submitted papers will not appear in the
CVPR proceedings. Thus, it is likely that the paper you have in your
hands will be refined further and submitted to some other journal or
conference, or even to CVPR next year. Sometimes the work is still
considered confidential by the authors' employers. These organizations
do not consider sending a paper to CVPR for review to constitute a
public disclosure. Protection of the ideas in the papers you receive
- You should not show the paper to anyone else, including colleagues
or students, unless you have asked them to write a review, or to help
with your review.
- You should not show any results or videos/images or any of the
supplementary material to non-reviewers.
- You should not use ideas from papers you review to develop new
- After the review process, you should destroy all copies of papers
and videos and erase any implementations you have written to evaluate
the ideas in the papers, as well as any results of those
As a reviewer of a CVPR paper, it is important for you to avoid any
conflict of interest. There should be absolutely no question about the
impartiality of any review. Thus, if you are assigned a paper where
your review would create a possible conflict of interest, you should
return the paper and not submit a review. Conflicts of interest
include (but are not limited to) situations in which:
While the organizers make every effort to avoid such conflicts in the
review assignments, they may nonetheless occasionally arise. If you
recognize the work or the author and feel it could present a conflict
of interest, email the Program Chairs as soon as possible so he or she
can find someone else to review it.
- You work at the same institution as one of the authors.
- You have been directly involved in the work and will be receiving
credit in some way. If you're a member of the author's thesis
committee, and the paper is about his or her thesis work, then you
- You suspect that others might see a conflict of interest in your
- You have collaborated with one of the authors in the past three
years (more or less). Collaboration is usually defined as having
written a paper or grant proposal together, although you should use
- You were the MS/PhD advisor of one of the authors or the MS/PhD
advisee of one of the authors. Most funding agencies and publications
typically consider advisees to represent a lifetime conflict of
interest. CVPR has traditionally been more flexible than this, but you
should think carefully before reviewing a paper you know to be written
by a former advisee, especially a recent one.
Belittling or sarcastic comments have no place in the reviewing
process. The most valuable comments in a review are those that help
the authors understand the shortcomings of their work and how they
might improve it. Write a courteous, informative, incisive, and
helpful review that you would be proud to add your name to (were it
The following provides further details for reviewing papers using the
CVPR 2016 submission and reviewer system. In addition, please see the
Reviewer Guidelines above and the
Once you've been notified that the papers have been assigned to you,
please log in to the CMT site and follow these steps:
- The submission/review site is
(bookmark or save this URL!)
- Please add "firstname.lastname@example.org" to your list of safe senders in
your own email client to prevent important email announcements from
being blocked by spam filters.
- Do not create a new account on CMT! If you need to change the
email address on your account, email the PCs and we will do it for
you. If you need to change only your name, you can do this in the CMT
system yourself using the same account.
- In the Section "Paper Reviews and Discussions", click on "Download
Assigned Papers". This allows you to download a zip file containing
all the papers plus supplementary files (if available).
- Contact the Program Chairs immediately if:
- You think you are in any way conflicted with the paper (see
- There is a violation of the stated paper submission rules.
- Such a violation includes:
- Over 8 pages (not including references),
- Constitutes a double submission,
- Supplementary material includes a newer version of the
- Please specify the exact nature of the violation.
- For your reference, the paper submission guidelines, which
includes descriptions of CVPR 2016's dual submission policy, can be
found in the Author
- "Paper Summary" label: next to it, you'll see the icons "+" and
"-". Clicking on "+" shows you all the abstracts; clicking on "-"
collapses all them back.
- At the end of each paper title, you'll see "+" as well. This has
the same function of showing the abstract for that paper, toggling to
"-" at the same time, which collapses it when selected.
- Please take the time to familiarize yourself with the table
entries; clicking on any of the column heading (e.g., "Paper ID")
sorts according to its description.
- For a paper, under the review column, click "Add" (to the right of
the "Review" line) to enter a review. Please see
the Reviewer Guidelines above and
take each review seriously. The review form contains detailed
instructions about writing the review and the meaning of the different
ratings (Oral, Poster, etc.).
- CMT does not allow users to type certain characters into a text
box that could be interpreted as html tags (for example, "y<x") or a
malicious script. As a workaround, introducing spaces between these
characters (for example, "y < x") will allow you to submit the text
since this can no longer be interpreted as an html tag.
- If you save your review as a draft, it is visible only to you. You
can access your draft review form by clicking on the same "Add"
link. To make the review visible to the Area Chair, click on the
"Submit" button in the review form. "Submit" won't work if any of the
required items is not filled.
- You have two options to access the "Offline Reviewing" page: (1)
In the "Paper Reviews and Discussions" page, click on "Review papers
offline" link near the top of the page, or (2) In the "View/Edit
Review" page, click on "offline reviewing" link.
- In the "Offline Reviewing" page, you can download one review
template file for a single paper, several papers, or all the
papers. We suggest that you download a review template file for each
paper to avoid confusion.
- Please read instructions on how to modify the file to incorporate
your responses. Note that you must not add certain characters in your
responses that could be interpreted as html tags or a malicious
script. See item 3 above.
- You can upload the completed file using the "Upload" interface at
the bottom of the page. The new uploaded version will (destructively)
overwrite the current review.
- We suggest that you try downloading a review template file for one
paper, enter test responses, and upload to get a sense of how it
- You should always verify the review after uploading (by inspecting
- We suggest that you use an XML editor to edit the file, for
example: EditiX (Windows, Unix/Linux, Mac OS X) or XML
Notepad 2007 (Windows only). (Remember to edit only fields
currently filled with the phrase "REPLACE THIS WITH YOUR
- After the rebuttal period, reviewers will work with Area Chairs to
clear up any confusions and attempt to reach consensus on papers.
- The CMT site has an electronic bulletin board feature that allows
Area Chairs to contact reviewers anonymously. Once the Area Chair
posts a note, reviewers will be notified and asked to log in to see
the post and respond. The identities of the reviewers will be hidden
from each other.
- After the rebuttal period you will enter your final
recommendation on CMT. This may differ from your preliminary
rating, and should reflect your judgment taking into account all the
other reviews, the authors' rebuttal, and the discussion about the
paper (if any).
- Is there a minimum number of papers I should accept or
No. Each paper should be evaluated in its own right. If
you feel that most of the papers assigned to you have value, you
should accept them. It is unlikely that most papers are bad
enough to justify rejecting them all. However, if that is the
case, provide clear and very specific comments in each review.
Do NOT assume that your stack of papers necessarily should have
the same acceptance rate as the entire conference ultimately
- Can I review a paper I already saw on arXiv and hence know who the authors are?
Yes.See next bullet below for guidelines. .
- How should I treat papers for which I know the authors?
Reviewers should make every effort to treat each paper fairly, whether or not they know who wrote the paper. For example:
- It is Not OK for a reviewer to read a paper, think "I know who wrote this; it's on arXiv; they're usually quite good" and accept paper based on that reasoning.
- Conversely, it is also Not OK for a reviewer to read a paper, think "I know who wrote this; it's on arXiv; they're no good" and reject paper based on that reasoning.
- How should I treat arXiv papers?
ArXiv papers are not considered prior work since they have not been peer reviewed. Therefore, you should review your CVPR papers independently as if the ArXiv papers didn't exist. Citations to these papers are not required and failing to cite or beat performance of arXiv papers are not grounds for rejection. For example:
- It is Not OK for a reviewer to suggest rejection for not citing an arXiv paper or not being better than something on arXiv.
- It is Not OK to accept a paper solely because it performs better than something on arXiv.
- It is Not OK to reject a paper solely because it performs worse than something on arXiv.
- It is Not OK to regard arXiv as a standard for the state of the art, because it is not reviewed. This applies *whoever* wrote the arXiv paper.
- It is Not OK for a reviewer to reject a paper solely because another paper with a similar idea has already appeared on arXiv. If the reviewer is worried about plagiarism they should bring this up in confidential comments to the AC.
- It is OK for a reviewer to suggest an author should acknowledge and be aware of something on arXiv.
- It is OK for an author to decline to acknowledge something on arXiv (because it has not been reviewed and so may not be right).