Great progress has been made over the last 10 years in the success of artificial neural networks (ANN). A seminal paper describing some of these was co-authored by Hinton Deep Learning, Yann LeCun, Yoshua Bengio and Geoffrey Hinton This paper has been presented in many forms we will consider three of them

Computing and Information Systems/Creative Computing CO3311 Neural networks
Coursework assignments 1 and 2 2019–2020

Important

Your coursework assignments should be submitted using the following file-naming conventions:

YourName_SRN_COxxxxcw#.pdf (e.g. GraceHopper_920000000_CO3311cw1.pdf)

• YourName is your full name as it appears on your student record (check your student portal).
• SRN is your Student Reference Number, for example 920000000.
• COXXXX is the course number, for example CO3311.
• cw# is either cw1 (coursework 1) or cw2 (coursework 2).

Any other parts of your submission (spreadsheets, code, etc.) should be submitted in a zip file that uses the same file-naming conventions: YourName_SRN_COxxxxcw#.zip

REMINDER: It is important that your submitted assignment is your own individual work and, for the most part, written in your own words. You must provide appropriate in-text citation for both paraphrase and quotation, with a detailed reference section at the end of your assignment (this should not be included in any word count). Copying, plagiarism and unaccredited and wholesale reproduction of material from books, online sources, etc. is unacceptable, and will be penalised (see our guide on how to avoid plagiarism on the VLE). You can also look at the end of any journal or conference paper to get an idea of how to cite your reference material appropriately.

Question 1

Great progress has been made over the last 10 years in the success of artificial neural networks (ANN). A seminal paper describing some of these was co-authored by Hinton:

“Deep Learning”, Yann LeCun, Yoshua Bengio and Geoffrey Hinton

This paper has been presented in many forms; we will consider three of them. For this question you are required to read the following three sources:
https://www.cs.toronto.edu/~hinton/absps/NatureDeepReview.pdf https://www.cs.tau.ac.il/~dcor/Graphics/pdf.slides/YY-Deep%20Learning.pdf http://www.personal.psu.edu/lxx6/DeepLearning.pdf

In an essay of about 1,000 words answer the following questions:

a) What concepts are introduced in these papers that extend those covered in the CO3311 subject guide and relevant readings?
b) Summarise each source.
c) Are these new concepts easy to understand given the material that you have already covered in CO3311?
d) What improvements would you have liked to see in these articles?

To answer these questions fully you are likely to require a search engine to find out more about the concepts that are mentioned in your answer to part (a). The material in Systematic evaluation of CNN advances on the ImageNet by Mishkin, Sergievskiy and Matas may be useful for this and for Question 2 below.

[20 marks]

In the articles given in Question 1 above, reference is made to a number of different activation functions. This question requires you to investigate what impact the use of these has on the accuracy and learning time of a single Backpropagation unit. In Question 2 of coursework assignment 2 you will be expected to take this further to investigate networks of three units, two in the hidden layer and one in the output layer.

a) Implement a 2-input single-unit Backpropagation network for each activation function that appears in the articles cited above.
b) Test your network with data for which you know the answers.
c) Train and evaluate your network on the data in Table 1 below. Collect data for the sum of square errors for each epoch after 0, 1, 10, 100, 1000 and 10000 epochs of training.
d) Carry out an analysis making a comparison of the results for each activation function in terms of how quickly the error reduces as a function of the number of epochs.
e) State any conclusions that can be made from your results.

Your answer should include:

• An introduction.
• A description of your program or spreadsheet, and how you developed and tested it, including an explanation of your test strategy.
• Sets of results as graphs of error vs epochs for each individual activation function as well as a graph with the results for all activations shown for comparison.
• An analysis of the results obtained including any surprising or interesting observations that you made.
• A conclusion.

For a good example of how to write up ANN experiments see:
Rectified Linear Units Improve Restricted Boltzmann Machines by Vinod Nair and Geoffrey Hinton (2010).

If you wish, you can modify the spreadsheet 2-sigmoid.xls, which is downloadable from the CO3311 course page on the VLE. If you prefer, you can use another piece of software or write your own.

In order to compare results from different functions, it is important to keep the scales on any graphs identical. Use log scales for both the x axis (number of epochs) and the y axis (sum of squared errors). Use the full width of a page (in portrait orientation) for each graph.

Table 1a Training data

Table 1b Validation data

Table 1c Evaluation data

[80 marks]

[Total 100 marks]

[END OF COURSEWORK ASSIGNMENT 1]

Question 1

Type ‘Deep learning’ into Google Scholar and find the following entry (this was the top result at the time of writing this coursework assignment):

Take note of the number of citations that the article has (which may differ when you do the search) as well as the number of versions.

By clicking on the number of citations, bring up a list of these and sort them by date with the newest first.

Many of the articles are freely available, however availability may depend on your location and whether you are at a teaching centre or not. The rightmost column of the results gives links that you can explore to see if you can obtain a free download. Browse these until you find two articles that you find readable and sufficiently detailed (giving technical details not just marketing material).

Write an account of an article showing:

i) Previous techniques used.
ii) Good progress using deep learning.
iii) Advantages of using deep learning.
iv) Any shortcomings of deep learning noted in the article.

Your essay (of about 1,000 words) should be structured as follows, and include the following sections with the appropriate headings:

• Introduction.
• Account of an article showing points i-iv (see above).
• Conclusions.
• References.

Remember to include citations and references for all sources used, and to do this in the Harvard format.
[25 marks]

This question requires you to extend the work done for Question 2 of coursework assignment 1 to the case of a 2-input 3-unit Backpropagation network (with two hidden units).

Table 2a contains a training set, Table 2b a validation set and Table 2c a test set. Your experiments must include the following:
a) Learning rates of 0.05, .1, .25, .5, .75 and 1.0.
b) Five replications for each set of learning rates, using different randomly selected initial values for all of the weights.
c) Plots of sum of square errors after 0, 1, 10, 100, 1000 and 10000 epochs. Use log scales for both x (number of epochs) and y (sum of errors) axes.
d) Discuss how the speed and accuracy achieved as a function of epochs varies between different activity functions.
e) Is one activity function clearly better than the others?

Table 2a Training set

Table 2b Validation set

Table 2c Test set

What to submit:

• A single PDF containing:

 A description of the measure of errors that you used to monitor progress in your experiments.
 A description of the way that you implemented the networks with enough detail to enable others to duplicate your work.
 Summary table(s) and graphs of your results
 A commentary on the results that you have obtained, describing any features that you feel are notable.
 Any conclusions that you can make.
 A list of references, using the Harvard system.

• A zip file including data files, program code or Excel spreadsheets for each of the experiments.

As for Question 2 of coursework assignment 1, you are advised to look at Rectified Linear Units Improve Restricted Boltzmann Machines by Nair and Hinton to see a good example of how ANN experiments can be written up.

[75 marks]

[Total 100 marks]

[END OF COURSEWORK ASSIGNMENT 2]

Acemyhomework
Calculate your paper price
Pages (550 words)
Approximate price: -
Acemyhomework

Quality Research Papers

We always make sure that our academic writers follow all your instructions precisely. You can choose your academic level and we will assign a writer who has a respective degree.

Acemyhomework

Best Academic Writers

We have a team of professional writers with experience in academic and business writing. Many are native speakers and able to perform any task for which you need help.

Acemyhomework

Unlimited Revisions

If you think we missed something, send your order for a free revision. You have 10 days to submit the order for review after you have received the final document.

Acemyhomework

On time Delivery

All papers are always delivered on time. In case we need more time to master your paper, we may contact you regarding the deadline extension. Otherwise a 100% refund is guaranteed.

Acemyhomework

Original & Confidential

We use several writing tools checks to ensure that all documents you receive are free from plagiarism. We also promise maximum confidentiality in all of our services.

Acemyhomework

24/7 Customer Support

Our support agents are available 24 hours a day 7 days a week and committed to providing you with the best customer experience. Get in touch whenever you need any assistance.

Try it now!

Calculate the price of your order

Total price:
$0.00

How it works?

Follow these simple steps to get your paper done

Homework Market

Place your order

Fill in the order form and provide all details of your assignment.

Homework Market

Proceed with the payment

Choose the payment system that suits you most.

Homework Market

Receive the final file

Once your paper is ready, we will email it to you.

Academic Writing Services

Whether you have an urgent deadline or those that have time. You can take some time and relax after trusting us with your paper. We make sure that we conduct the academic writing services diligently.

Acemyhomework Acemyhomework

Essays

Essay Writing Service

Among the wide variety of academic work, essay writing is one of the simplest a student can ever come across. Usually, it is a task which students encounter and learn how to write whilst in high school. However, the case is quite different when it comes to university and college.

Acemyhomework Acemyhomework

Term Papers

Term Paper Writing

Are you looking for an online writing firm that can offer you reliable custom term paper writing help? Is your wish and desire to get someone who can guide you throughout the process of writing term papers? If yes, then you have come to the right place.

Acemyhomework Acemyhomework

Coursework

Coursework Writing Help

Coursework is essential for every student in order to graduate from college. However, most of it is deadline-centric, and that becomes a challenge to most learners. With the amount of work, learners are receiving every day, finding time to work on every task is not easy.

Acemyhomework Acemyhomework

Homework

Online Homework Help

Online homework help services are an answer to every challenge that students go through. Despite the difference in the needs and levels of learning, all students can benefit from these services. Acemyhomework is one of the best online homework help service companies you can find on the internet.