Agenda and Minutes
1. Updates and plans.
- Let's look at MH's reviews to analyze and determine what to do. The response document we will send to the editor will be basically the following:
Response: <MH had to omit a lot of the time series model building process due to lack of space. The response is to put back the necessary and sufficient material. Then just write "We added in the requested material. Please see section ___ for full details." Doesn't have to be everything but enough. Page charges are just something to deal with later.>
Reviewer #3:
Major issues:
1.
On page 2, the basic information of the data should be (re)stated in a clear
way in this paper, even though the data was used in the previous literature.
Basic information like the source of the data, the size of the data, the time
range, etc. could be introduced.
Response:
<Add that information back in. Say "We added in the requested information in section _____.">
Reviewer #3:
2.
In the random walk with drift model (on page 3), it is better to use 𝜀𝑡, rather than 𝑛𝑡, to denote the white noise.
Response:
<Fix it, and in the response, just say "done.">
Reviewer #3:
3.
In the last paragraph on page 3, the paper claimed the forecast was based on an
ARIMAX model, but no predictor variables (or the regression part) could be
identified in the paper.
Response:
<Fix this, and in the response, just say "done.">
Reviewer #3:
4.
On page 4, the R packages used should be explicitly listed out, at least the
most important ones.
Response:
<Fix this, and in the response, just say "done.">
Reviewer #3:
5.
On page 4, there is a major issue with applying Augmented Dickey-Fuller (ADF)
test to identify stationary in this paper. An ADF test could be used only under
the assumption that the time series follows AR(p) model. However, the
paper didn’t verify the time series that follows AR(p) model before
applying the test. What’s more, if it is an AR(p) model, the author
should also estimate p, the approximating AR order, based on some
information criteria (for example, AIC, BIC) before carrying out the ADF test.
When applying the ADF test, why 5 lags was chosen should be explained.
Response:
<Find where the reviewer might have decided on this criticism, and add a note at that location that the explanation is later. In the response, write "We fixed the text to clarify early in the paper that a fuller explanation is given later in the paper.">
Reviewer #3:
6.
In the paragraph above Figure 5 on page 4, the number of data points (not
just the percentages) that were used to produce the model should be stated.
Response:
<Fix it and then respond, "Done.">
Reviewer #3:
Reviewer #3:
8.
As the issue mentioned in item 8 above, the parameters chosen for ARIMAX(p,
d, q) were not explained.
Response:
<Explain what you mentioned in the meeting in the paper, and respond that you added that explanation into the paper. "We added the following passage into the paper: __________">
Reviewer #3:
9. Several key steps in the model building process were skipped in the paper. For choosing a time series model, the common practices include plotting the time series, examining ACF and PACF plots. After the model is identified, the model should be examined and follow the diagnostic check, before making forecasts. The analysis in the paper skipped several major steps, which could result in unreliable forecasts.
For example, a similar model
building procedure in the following two papers is recommended:
•
• Fang-Mei Tseng,
Gwo-Hshiung Tzeng, A Fuzzy Seasonal ARIMA Model for Forecasting, Fuzzy
Sets and Systems, 126 (2002), 367~376
•
• Lazim Abdullah,
ARIMA Model for Gold Bullion Coin Selling Prices Forecasting, International
Journal of Advances in Applied Sciences, Vol. 1, No. 4, December 2012, pp.
153~158
Response:
<We added the suggested information to the paper in section ______.>
Reviewer #3:
Reviewer #3:
11. In Figure 6 and Figure 7,
why the logarithm of years was used should be explained.
Response:
<Fix and give the new text here in the response.>
Reviewer #3:
Minor issues:
1.
There are some mismatches between authors’ names and institutional
affiliations.
Response:
<Check on this - what's the problem? Then respond "Done." RS points out that the name of his dept. has changed recently. The new name is "Information Systems & Business Analytics" (ISBA). Then respond "Done.">
Reviewer #3:
2.
In the first sentence in section Methods and Results, the stated “1.3”
could not be located in the paper.
Response:
<Fix and say "Done.">
Reviewer #3:
3.
The font size of the whole paper should be unified.
Response:
<Fix and say "Done.">
Response:
<Fix and say "Done.">
Reviewer #7:
Response:
<We removed the "recent phenomenom" claim and left in place our citations to other researcher as well as adding in reference __________.>
Reviewer #7:
The authors used an old graph and only showed the data from 2001 to 2019 in Figure-1, while there is an updated graph showing the data from 2001 to 2020 Sequencing Costs 2020.
Response:
<Delete the graph and just give an updated reference, and respond "We deleted the graph but updated the reference as suggested so that readers can check for themselves.">
Reviewer #7:
The same applies for Figure-2, there is an updated version. It is clear that all these examples show exponential behavior. Instead of using these figures in a research paper, the authors could just cite to the source of the figures and briefly explain the exponential technologies.
Response:
<Fix and respond with, "Done.">
Reviewer #7:
The authors are interested in finding trends in space exploration technology using the lifespan as a metric and then conclude the “Progress in Space Technology” section that this metric is problematic and required other modeling techniques. In next section, the authors revisited the forecast distribution model and its source of uncertainties with a focus on the impact of random shocks. Farmer and Lafond (2016) method is explained with its advantages/disadvantages and the need of modeling other than the random walk model is described. The section on “Determining Forecast Quality” is finalized with referencing the section 2, and “Methods and results” is started with referencing to the section 1.3. In the paper, the sections and subsections are not numbered. It is not possible to follow and understand the method which is based on time.
Response:
<Number the sections, and say that the numbers are now added. If there was a mistake you could say that an error in numbering might have confused the reader and this has been fixed.>
Reviewer #7:
Overall, the conclusions are not supported by the data comparison. It is not possible to make a conclusion which method is the best to model improvements in technological performance.
Response:
<Add a sentence or two to the conclusions about which method is best. Write "Our many updates to the paper should help to clarify the argument, however, if the reviewer still has some concerns we would be happy to make further updates as requested." "We supported our conclusion with an additional explanatory passage.">
Reviewer #7:
This paper is not well-written.
The font size changes throughout the text.
There are so many grammar mistakes, starting from the abstract
e.g:
• “be successful modeled” -> “be
successfully modeled”,
• “a approach” -> “an approach”
There are footnotes (law-4 on page1, model-1 on page3) which are
not used.
The sections and subsections are not numbered, but in the paper
there are
references to
the sections.
Response:
<"We thank the reviewer for pointing out these needed improvements and we carefully fixed the issues.">
Reviewer #8:
The authors established inadequacy of random walk-based model for spacecraft lifespan and fitted two different time-series models on this data. I have the following points about the content and presentation of statistical analyses.
(a) The authors accurately argued about inadequacy of random-walk model using the autocorrelation seen in Figure 05. I think, it will also be interesting to include a diagram of backtesting, similar to those in Figures 6 and 7, but obtained using a random-walk with drift model. It can help the reader visually identify how an incorrect choice of model influences prediction at withheld points.
Response:
<"We clarified the discussion of Figure ___ which addresses this point. We expanded the explanation of __________. (If you add in more text you could quote it here in the response, if it seems good to do.>
Reviewer #8:
(b) Please describe both ARIMA(1,1,0) and ARIMAX(1,0,0) and their output in greater detail. This will include (i) specifying the time series equation for each of them, (ii) reporting the numerical values of all parameter estimates, (iii) comparing the estimates of the autoregressive parameter between the two models.
Response:
<Thank you for pointing out the need for these. We added the requested information in section _____.>
Reviewer #8:
(c) The authors mentioned that they have used “various R statistical packages’’. Please provide the details on which packages were used and cite them appropriately.
Response:
<"Done.">
Reviewer #8:
(d) For both of Figures 6 and 7, the authors did not describe what the blue line represents and what the wider, grey-colored interval signifies. What is the difference between blue interval and grey interval in each of these two diagrams? Are these 90% or 95% intervals?
Response:
<We reorganized and clarified the text to answer these questions better.>
Reviewer #8:
(e) Please report the findings from unit root test in terms of test statistics and/or p-value.
Response:
<"Done.">
Reviewer #8:
(f) The authors mentioned parameter uncertainty as one source of forecast uncertainty. Have the authors considered use of hierarchical Bayes framework for implementing these time series models so that the parameter uncertainty is naturally taken into account in point prediction as well as interval prediction?
Response:
<This is a great suggestion. Although it is out of scope for the current paper we are considering this path for a future article.>
Reviewer #8:
(g) Figure 7 is incorrectly captioned as Figure 6.
Response:
<"Done.">
------------------------------------------------------------------------------------
(We adjourned at this point. The following items were not covered.)
------------------------------------------------------------------------------------
- Status of registration and travel fees. As of 6/25/2021, RS has paid $640 for WMSCI and applied for reimbursement from his dept. which has agreed to reimburse him for it. As of 6/25/21, PT paid $400 for FTC and the college has agreed to reimburse him for it.
2. Background literature update.
https://gizmodo.com/the-last-images-from-doomed-space-probes-1847100494
- Some interesting videos are at the Kartik Gada channel such as at https://www.youtube.com/channel/UCuRX67CJhaOT98Jdjh85CEQ which we discussed previously.
- For general reference here are some generic questions about articles (and videos):
a. What is the source?b. What is the most significant advance in the human knowledge presented in the paper?c. Why is that advance important?d. What important questions arise from the paper for future research?e. What important questions would it be nice if the paper answered, but does not answer?f. What does the paper present that is novel (no one else has provided that before)?g. What is the relevance of the paper to our satellite research goals?h. Questions from the group?
No comments:
Post a Comment