Data Analysis Using Python: Tools and Techniques Assignment Sample

Introduction OfData Analysis Using Python: Tools and Techniques

In the introduction of data analysis using Python is a more powerful and various methods of extracting insights from data. The Pandas and Numpy is used for various types of libraries that offer the most efficient data manipulation process and used some statistical data analysis tools for making a huge choice for professions in various industries.

Task 1: High-Frequency Finance

1.1

F1:

Figure 1: Removing entries outside the normal market

(Source: self-created in Google Colab)

The above figure shows the removal of the entries outside in the normal market. This is self-created in Google Colab. In Python removing entries filtering out for the data points that is particular from a dataset that is removing beyond a particular range or criteria that should be referred to as “normal”.

F2:

The above images show the actual entries and the removal of the prices. In python first particularly the specific entries that is created from the that is using “remove ()” methods and it should be used to analyze for filtering the particular, this is a comprehension and exclude the entries the want to remove.

F3:

For removing the entries with bid-asked price that should be shown here. To remove the various types of entries with the bid-asked price in Python that can be used in the list of comprehension and it should filter out the various types of elements that have been used for a non-zero bid asked price.

F4:

The aggregating entries in the Python that also refers to combining and merging the data from the various sources into a single summarized dataset. This can be done using different procedures like concatenation, merging, or appending (Captier et al. 2022). That also utilizes the funations using various types of commands and here it is used as various libraries such as pandas that can achieve this. For aggregating the particular entries that can effectively analyze, manipulate and summarize data from various data from different origins for future analysis or processing.

F5:

The removal of the various types of entries with bid-ask spread is more than 50 times that means using filters to filter out the particular data where the actual difference between the buying price (bid) and selling price (ask) is excessively high.

F6:

The creating the data frame that shows the number and the previous portion of entries removed from the dataset that should be used in Google Colab. Huge datasets are also included here.

Task 2: Return-Volatility Modelling

2.1

Best-fitted ARMA (AutoRegressive Moving Average) models capture temporal dependencies and random fluctuations in data, making them useful in time series analysis and forecasting. They combine autoregressive and moving average components for optimal performance.

2.2

Row 1:

i)

Figure 8: Time series plot for standardized residuals

(Source: self-created in Google Colab)

Import matplotlib.pyplot and arch. Fit GARCH model to return data. Get standardized residuals using model.standardized_residuals(). Plot residuals over time using plt.plot(residuals). Add title and labels. Check for visible patterns. Residuals should appear random over time if the model fits well. Assess model fit visually.

ii)

Figure 8: Histogram for standardized residuals

(Source: self-created in Google Colab)

Import matplotlib.pyplot and arch. Fit GARCH model to return data. Get standardized residuals using model.standardized_residuals() (Chen, 2019). Plot a histogram of residuals using plt.hist(residuals, bins=20). Add labels and title. Histogram should be bell-shaped if residuals are normally distributed. Assess normality visually.

iii)

Figure 9: ACF of standardized residuals

(Source: self-created in Google Colab)

Import arch and statsmodels. Fit GARCH model to return data. Extract standardized residuals using model.standardized_residuals(). Calculate ACF of standardized residuals using sm.tsa.acf(). Plot ACF over lags. Check for significance. White noise residuals should have no significant autocorrelation.

iv)

Figure 10: ACF of squared standardized residuals

(Source: self-created in Google Colab)

Import arch and statsmodels. Fit GARCH model to return data. Get standardized residuals using model.standardized_residuals(). Square the standardized residuals. Calculate ACF of squared residuals using sm.tsa.acf(). Plot ACF over lags. Significant autocorrelation indicates volatility clustering remaining.

Row 2:

i)

Figure 11: Time series for the second stock

(Source: self-created in Google Colab)

Import matplotlib.pyplot and pandas. Load second stock price data into dataframe df. Set dataframe index to DateTime. Plot close price over time using plt.plot(df['Close']) to create line chart. Customize chart labels, legend, title. Show plot. Visualizes price movement over time.

ii)

Figure 12: Histogram for the second stock

(Source: self-created in Google Colab)

Import matplotlib.pyplot and numpy. Load returns data for second stock into returns variable. Calculate histogram of returns using plt.hist(returns, bins=20), specifying 20 bins (Fouché et al. 2023). Add labels and title to Show plot. The histogram visualizes the distribution of stock returns.

iii)

Figure 13:

(Source: self-created in Google Colab)

Import arch module. Fit GARCH model to returns of second stock. Extract standardized residuals using model.standardized_residuals(). Calculate ACF of standardized residuals using sm.tsa.acf(). Plot ACF over time lags. Assess whether residuals are white noise by looking for significant autocorrelation.

iv)

Import arch module. Fit GARCH model to returns of second stock. Standardize residuals using model.standardized_residuals. Square standardized residuals. Calculate ACF of squared residuals using sm.tsa.acf. Plot ACF over time lags to assess residual autocorrelation and ARCH effects remaining.

2.3

Import arch module. Standardize residuals of GARCH model fit to second stock. Square standardized residuals. Calculate ACF of squared standardized residuals using sm.tsa.acf function. Plot ACF to visualize autocorrelation in squared residuals over time lags.

2.4

i)

Figure 16: Time series for standardized residuals

(Source: self-created in Google Colab)

Using matplotlib to create a time series plot of the residuals from an ARIMA or Prophet model that have been standardized by dividing by their standard deviation, checking for patterns like autocorrelation that violate model assumptions.

ii)

Figure 17: Histogram for standardized residuals

(Source: self-created in Google Colab)

Using matplotlib to plot a histogram of residuals from a regression model that have been standardized by dividing by their standard deviation, checking that they follow a normal distribution to validate regression assumptions.

2.5

Import arch module. Create GARCH model specifying mean and volatility equations. Fit model to return data. Extract fitted volatility estimates using fitted values attribute of model. Plot fitted volatility over time.

Task 3: Return-Volatility Forecasting

3.1

Making predictions for the next time step using previous observations and a model like ARIMA or Prophet in Statsmodels, useful for short-term forecasts and quantifying uncertainty without accounting for future shocks or causal factors.

3.2

Using scipy.stats to calculate the range between the 2.5th and 97.5th percentile of a sample distribution, indicating the interval within which we can be 95% confident the true population parameter lies based on the observed data.

3.3

One-step forecasts produced from analytic time series models like ARIMA or ETS in Statsmodels based solely on past data, useful for short-term predictions without accounting for future shocks or domain knowledge about causal factors.

3.4

Using autoregressive models like ARIMA in Statsmodels to predict next-period returns based on lagged values, helping quantify uncertainty in short-term predictions for financial time series analysis and trading strategies.

3.5

Using pandas and stats models to calculate 95% confidence intervals for time series forecasting models like ARIMA or Prophet, providing a range that the true value will fall within 95% of the time.

Conclusion

In conclusion, of the data analysis that is conducted using Python gives valuable insights, for revealing significant trends and patterns. The findings will contribute to informed decision-making and extract the overall understanding of the data.

References

Journals

• Captier,N., Merlevede,J., Molkenov,A., Seisenova,A., Zhubanchaliyev,A., Nazarov,P.V., Barillot,E., Kairov,U. and Zinovyev,A. (2022) BIODICA: a computational environment for Independent Component Analysis of omics data. Bioinformatics, 38, 2963–2964.
• Chen,S., Lake,B.B. and Zhang,K. (2019) High-throughput sequencing of the transcriptome and chromatin accessibility in the same cell. Nat. Biotech., 37, 1452–1457.
• Fouché A, Chadoutaud L, Delattre O, Zinovyev A.Transmorph: a unifying computational framework for modular single-cell RNA-seq data integration. NAR Genom Bioinform. 2023 Jul 12;5(3):lqad069. doi: 10.1093/nargab/lqad069. PMID: 37448589; PMCID: PMC10336778.
• irkes,E.M., Bac,J., Fouche,A., ´ Stasenko,S.V., Zinovyev,A. and Gorban,A.N. (2022) Domain adaptation principal component analysis: base linear method for learning with out-of-distribution data. Entropy,
• Krismer E, Bludau I, Strauss MT, Mann M. AlphaPeptStats: an open-source Python package for automated and scalable statistical analysis of mass spectrometry-based proteomics. Bioinformatics. 2023 Aug 1:btad461. doi: 10.1093/bioinformatics/btad461. Epub ahead of print. PMID: 37527012.
• Miller,H.E., Gorthi,A., Bassani,N., Lawrence,L.A., Iskra,B.S. and Bishop,A.J. (2020) Reconstruction of Ewing sarcoma developmental context from mass-scale transcriptomics reveals characteristics of EWSR1-FLI1 permissibility. Cancers, 12, 948.
• Sompairac,N., Nazarov,P.V., Czerwinska,U., Cantini,L., Biton,A., Molkenov,A., Zhumadilov,Z., Barillot,E., Radvanyi,F., Gorban,A. et al. (2019) Independent component analysis for unraveling the complexity of cancer omics datasets. Int. J. Mol. Sci., 20, 4414.
• Zhou Y, Yang D, Yang Q, Lv X, Huang W, Zhou Z, Wang Y, Zhang Z, Yuan T, Ding X, Tang L, Zhang J, Yin J, Huang Y, Yu W, Wang Y, Zhou C, Su Y, He A, Sun Y, Shen Z, Qian B, Meng W, Fei J, Yao Y, Pan X, Chen P, Hu H. Author Correction: Single-cell RNA landscape of intratumoral heterogeneity and immunosuppressive microenvironment in advanced osteosarcoma. Nat Commun. 2021 Apr 30;12(1):2567. doi: 10.1038/s41467-021-23119-7. Erratum for: Nat Commun. 2020 Dec 10;11(1):6322. PMID: 33931654; PMCID: PMC8087801.