1. Updates and plans.
Response: <MH had to omit a lot of the time series model building process due to lack of space. The response is to put back the necessary and sufficient material. Then just write "We added in the requested material. Please see section ___ for full details." Doesn't have to be everything but enough. Page charges are just something to deal with later.>
Reviewer #3:
Major issues:
1.
On page 2, the basic information of the data should be (re)stated in a clear
way in this paper, even though the data was used in the previous literature.
Basic information like the source of the data, the size of the data, the time
range, etc. could be introduced.
Response:
<Add that information back in. Say "We added in the requested information in section _____.">
Reviewer #3:
2.
In the random walk with drift model (on page 3), it is better to use 𝜀𝑡, rather than 𝑛𝑡, to denote the white noise.
Response:
<Fix it, and in the response, just say "done.">
Reviewer #3:
3.
In the last paragraph on page 3, the paper claimed the forecast was based on an
ARIMAX model, but no predictor variables (or the regression part) could be
identified in the paper.
Response:
<Fix this, and in the response, just say "done.">
Reviewer #3:
4.
On page 4, the R packages used should be explicitly listed out, at least the
most important ones.
Response:
<Fix this, and in the response, just say "done.">
Reviewer #3:
5.
On page 4, there is a major issue with applying Augmented Dickey-Fuller (ADF)
test to identify stationary in this paper. An ADF test could be used only under
the assumption that the time series follows AR(p) model. However, the
paper didn’t verify the time series that follows AR(p) model before
applying the test. What’s more, if it is an AR(p) model, the author
should also estimate p, the approximating AR order, based on some
information criteria (for example, AIC, BIC) before carrying out the ADF test.
When applying the ADF test, why 5 lags was chosen should be explained.
Response:
<Find where the reviewer might have decided on this criticism, and add a note at that location that the explanation is later. In the response, write "We fixed the text to clarify early in the paper that a fuller explanation is given later in the paper.">
Reviewer #3:
6.
In the paragraph above Figure 5 on page 4, the number of data points (not
just the percentages) that were used to produce the model should be stated.
Response:
<Fix it and then respond, "Done.">
Reviewer #3:
7.
There is a major issue with the ARIMA(p, d, q) model parameter chosen.
It could be easily seen from the paper why d = 1 was chosen. However, no
explanation about how p or q was chosen was given in the paper.
Response:
<Either reorganize the paper a bit, or put in a note that this will be explained later.>
Reviewer #3:
8.
As the issue mentioned in item 8 above, the parameters chosen for ARIMAX(p,
d, q) were not explained.
Response:
<Explain what you mentioned in the meeting in the paper, and respond that you added that explanation into the paper. "We added the following passage into the paper: __________">
Reviewer #3:
9. Several key steps in the
model building process were skipped in the paper. For choosing a time series
model, the common practices include plotting the time series, examining ACF and
PACF plots. After the model is identified, the model should be examined and
follow the diagnostic check, before making forecasts. The analysis in the paper
skipped several major steps, which could result in unreliable forecasts.
For example, a similar model
building procedure in the following two papers is recommended:
•
• Fang-Mei Tseng,
Gwo-Hshiung Tzeng, A Fuzzy Seasonal ARIMA Model for Forecasting, Fuzzy
Sets and Systems, 126 (2002), 367~376
•
• Lazim Abdullah,
ARIMA Model for Gold Bullion Coin Selling Prices Forecasting, International
Journal of Advances in Applied Sciences, Vol. 1, No. 4, December 2012, pp.
153~158
Response:
<We added the suggested information to the paper in section ______.>
Reviewer #3:
10.
On page 6, citing Wikipedia as a source should be avoided. The authors should
look for reliable sources such as journal articles or published books as cited
literature.
Response:
<Do that, and respond, "We added the reference ______________ and cited it instead.">
Reviewer #3:
11. In Figure 6 and Figure 7,
why the logarithm of years was used should be explained.
Response:
<Fix and give the new text here in the response.>
Reviewer #3:
Minor
issues:
1.
There are some mismatches between authors’ names and institutional
affiliations.
Response:
<Check on this - what's the problem? Then respond "Done." RS points out that the name of his dept. has changed recently. The new name is "Information Systems & Business Analytics" (ISBA). Then respond "Done.">
Reviewer #3:
2.
In the first sentence in section Methods and Results, the stated “1.3”
could not be located in the paper.
Response:
<Fix and say "Done.">
Reviewer #3:
3.
The font size of the whole paper should be unified.
Response:
<Fix and say "Done.">
Reviewer #3:
4. The figure on page 5 should
be labeled as Figure 7.
Response:
<Fix and say "Done.">
Reviewer #7:
MS#3414: This paper discusses which method to use to model the
technological advancement based on the analysis of mission lifespan data.
Forecast distribution (random walk with drift) doesn’t occur in space
exploration, and the authors proposed to show an alternative approach. In the “Introduction” section, the authors say “the recognition
that technology progresses in a predictable way” is a relatively recent
phenomenon. Actually it is not. Please cite to recent articles, where this phenomenon
has been attributed by many researchers.
Response:
<We removed the "recent phenomenom" claim and left in place our citations to other researcher as well as adding in reference __________.>
Reviewer #7:
The authors used an old graph and
only showed the data from 2001 to 2019 in Figure-1, while there is an
updated graph showing the data from 2001 to 2020 Sequencing Costs 2020.
Response:
<Delete the graph and just give an updated reference, and respond "We deleted the graph but updated the reference as suggested so that readers can check for themselves.">
Reviewer #7:
The same applies for Figure-2, there is an updated version. It is clear
that all these examples show exponential behavior. Instead of using these figures
in a research paper, the authors could just cite to the source of the
figures and briefly explain the exponential technologies.
Response:
<Fix and respond with, "Done.">
Reviewer #7:
The authors are interested in finding trends in space exploration
technology using the lifespan as a metric and then conclude the “Progress in
Space Technology” section that this metric is problematic and required
other modeling techniques. In next section, the authors revisited the forecast distribution
model and its source of uncertainties with a focus on the impact of random
shocks. Farmer and Lafond (2016) method is explained with its
advantages/disadvantages and the need of modeling other than the random walk model is
described. The section on “Determining Forecast Quality” is finalized with
referencing the section 2, and “Methods and results” is started with referencing
to the section 1.3. In the paper, the sections and subsections are not numbered.
It is not possible to follow and understand the method which is based on
time.
Response:
<Number the sections, and say that the numbers are now added. If there was a mistake you could say that an error in numbering might have confused the reader and this has been fixed.>
Reviewer #7:
Overall, the conclusions are not supported by the data comparison.
It is not possible to make a conclusion which method is the best to model improvements in technological performance.
Response:
<Add a sentence or two to the conclusions about which method is best. Write "Our many updates to the paper should help to clarify the argument, however, if the reviewer still has some concerns we would be happy to make further updates as requested." "We supported our conclusion with an additional explanatory passage.">
Reviewer #7:
This paper is not well-written.
The font size changes throughout the text.
There are so many grammar mistakes, starting from the abstract
e.g:
• “be successful modeled” -> “be
successfully modeled”,
• “a approach” -> “an approach”
There are footnotes (law-4 on page1, model-1 on page3) which are
not used.
The sections and subsections are not numbered, but in the paper
there are
references to
the sections.
Response:
<"We thank the reviewer for pointing out these needed improvements and we carefully fixed the issues.">
Reviewer #8:
The authors established inadequacy of random walk-based
model for spacecraft lifespan and fitted two different time-series models on this
data. I have the following points about the content and presentation of
statistical analyses.
(a) The authors accurately argued about inadequacy of
random-walk model using the
autocorrelation seen in Figure 05. I think, it will also be interesting to include a diagram
of backtesting, similar to those in Figures 6 and 7, but obtained using
a random-walk with drift model. It can help the reader visually identify
how an incorrect choice of model influences prediction at withheld points.
Response:
<"We clarified the discussion of Figure ___ which addresses this point. We expanded the explanation of __________. (If you add in more text you could quote it here in the response, if it seems good to do.>
Reviewer #8:
(b) Please describe both ARIMA(1,1,0) and ARIMAX(1,0,0) and
their output in greater detail.
This will include (i) specifying the time series equation for each of them,
(ii) reporting the numerical values of all parameter estimates, (iii)
comparing the estimates of the autoregressive parameter between the two
models.
Response:
<Thank you for pointing out the need for these. We added the requested information in section _____.>
Reviewer #8:
(c) The authors mentioned that they have used “various R
statistical packages’’. Please provide the
details on which packages were used and cite them appropriately.
Response:
<"Done.">
Reviewer #8:
(d) For both of Figures 6 and 7, the authors did not
describe what the blue line represents and
what the wider, grey-colored interval signifies. What is the difference between
blue interval and grey interval in each of these two diagrams? Are
these 90% or 95% intervals?
Response:
<We reorganized and clarified the text to answer these questions better.>
Reviewer #8:
(e) Please report the findings from unit root test in terms
of test statistics and/or p-value.
Response:
<"Done.">
Reviewer #8:
(f) The authors mentioned parameter uncertainty as one
source of forecast uncertainty. Have
the authors considered use of hierarchical Bayes framework for implementing
these time series models so that the parameter uncertainty is naturally taken
into account in point prediction as well as interval prediction?
Response:
<This is a great suggestion. Although it is out of scope for the current paper we are considering this path for a future article.>
Reviewer #8:
(g) Figure 7 is incorrectly captioned as Figure 6.
Response:
<"Done.">
------------------------------------------------------------------------------------
(We adjourned at this point. The following items were not covered.)
------------------------------------------------------------------------------------
2. Background literature update.
