CS E6998-10. Advanced Topics in Network Storage Systems 
Spring 2004

Grading criteria for HW #1


Problem 1.  10 pts

Grading was relaxed here.  The only requirements were that you specify
all the info required by the reviewer form and you provided a logical
analysis.

We generally expected you to offer some criticism, particularly in the
area of experimental methodology.  If you found absolutely nothing
wrong with the paper, you needed to be pretty convincing.  I also 
took a point off if the review was very hard to understand.


Problem 2-a.  5 pts

  Breakdown:

  Improves performance (+3) but only if the client request stream
  has sufficient locality or if there is sharing between multiple
  clients, or both.  (+2)

Problem 2-b.  15 pts

  Breakdown:

  Correctly identified the complementary relationship between
  the server and client cache.  The goal is to avoid having redundant
  file copies in both caches. (+5)

  Server cache should use MRU for pages that are brought in from the
  disk (promoted) and LRU for pages that are brought in from the
  client cache (demoted).  This minimizes redundancy between the
  client and server caches. (+5)  [Note: I accepted other solutions
  as long as the explanation took into account cache redundancy]

  Pointed out effect of multiple vs single clients.  The server cache
  replacement policy can be influenced by the level of file sharing
  achieved by multiple clients. (+5)

Problem 3.  20 pts

  Breakdown:

  No experimental setup.  You were asked to describe the operating
  environment used for the experiments (e.g. OS, CPU, memory).  Your
  results cannot be properly interpreted without this. (-2)

  Not enough test runs.  You should have run the tests several times
  for each block size to find the average times. (-4)

  No statistics for multiple test runs.  In addition to the average
  (mean) we also expected calculation of standard deviation or
  variance. (-2)

  No conclusion.  It is not enough to just provide the raw data.  You
  should provide a summary.  For instance, was SHA-1 a constant,
  linear, or exponential function of the block size?  How did the
  results compare with your predictions?  (-5)

  No graph. (-2)

  Graph or tables are improperly labeled (e.g. no axis labels or
  missing units) or hard to understand. (-1)

  Elapsed time includes disk I/O time.  Some experiments read files
  from disk instead of generating the data in memory.  This added a
  significant disk I/O time to the results.  Because the network
  bandwidth measurements do not include disk I/O, the comparison
  with SHA-1 time is invalid.  Anyone reading this paper would have
  no choice but to dismiss the results.  (-2)

  Elapsed time includes page fault time.  It is important to
  realize how to properly isolate the elapsed time to only measure
  the SHA-1 calculation time.  The best way to do this is to ensure
  that your data block has been paged into memory prior to the
  calculation.  Many people got this right by allocating the memory,
  randomizing each byte (thereby paging each page into memory),
  starting the timer, calculating the digest, and then stopping the
  timer.  (-1)