Principal applicant

Dr. Donal Fitzpatrick

School or Unit

School of Computing

Amount requested

15000

Summary

Education for the visually impaired continues to be a matter of concern. The problem spans almost all aspects of education, but is most notable in the study of STEM (Science, Technology, Engineering and Mathematics) disciplines. Through the use of technologies developed in current and previous research, we intend to develop a set of tools to enable communication of mathematical material between teacher and print-impaired student.

We aim to develop tools which will facilitate a teacher of STEM producing mathematical material and translating it to Braille or synthetically spoken output. It is not widely known that over 90% of blind people cannot read Braille, hence the choice of spoken output as an alternative presentational modality. With the evolution of refreshable Braille Displays comes the facility for a student to enter mathematics in the traditional, linear fashion. We also intend the development of so- called "back-translators" from this linear Braille representation to standard printed mathematics.

Project Url

www.computing.DCU.ie/~dfitzpat/multi-modal/Math

Will the project facilitate flexible access (in the context of lifelong learning or otherwise) or facilitation of longer-term provision for special needs

yes

Is the project related to the development of DCU's six academic themes

yes

Strategic Context

As theme II of the DCU Strategic plan states, "... obstacles faced by people who are socially disadvantaged, or who have disabilities, in accessing learning and education will be addressed." One of the major problems facing students who are

blind is accessing material such as mathematics. This project will endeavour to remove these obstacles, by providing a flexible mechanism for the exchange of mathematical and other scientific material between the lecturer and the student. As this theme also

advocates the development of "enabling technology", the proposed research would seem to be ideally suited for the furtherance of this theme. Another goal of the Strategic plan is to provide "... broader access to learning for all - including those with

disabilities.", we believe that this work will go some way to fulfilling this ideal.

Related work was supported by DCU Teaching and learning fund 2003/2004 as documented at:

http://odtl.DCU.ie/tlf/2003-2004/prj-0-awards/ref20/

Implementation Plan

The principal investigator and two research assistants will implement the project. The principal investigator will direct the work, and also engage in the more challenging aspects of the development process.

The first phase of this project will investigate the efficacy of the vocal characteristics used to convey the mathematical material, as outlined in [2,3]. Simultaneously, work will continue on the compilation of a set of rules whereby the linear Braille representation can be mapped to MathML.

In phase 2 work will commence on development. At this point, one research assistant will develop a tool which will parse the MathML into an internal format from which the synthetic speech can be derived, while the other will be responsible for the design of the interface; ensuring that it adheres to established and pertinent standards of accessibility. The Principle Investigator will be responsible for the highly complex and technical task of translating the linear Braille math notation back to the internal format.

Phase three will involve the integration of the various modules into a prototype system which can then be deployed and tested

The principles on which this work is based have been described by members of the International Group for Universal Math Access in a number of articles [1,2,3]. The goal of this ongoing collaborative work is to provide mathematical and scientific access to print-impaired students.

We envisage this project beginning on July 1 2005, and being completed by December 31 2006. No obstacles are foreseen which

will preclude meeting these targets.

[1] Archambault D., Batu?ić M., Berger F., Fitzpatrick D., Miesenberger K., Moço V., Stöger B., ?The Universal Maths Conversion Library: an attempt to build an Open Software Library to Convert Mathematical Contents in various Formats?, Proceedings of the 2005 Human Computer Interface International Conference, Las Vegas, USA, July 2005 (in press)

[2] Fitzpatrick D. Karshmer A.I.C., ?Multi-Modal Mathematics: conveying math using

synthetic speech and speech recognition?,, Proceedings of ICCHP 2004, Paris, France, July 2004

[3] Karshmer, A.I., and Gillan, D., "How well can we Read Equations to Blind Mathematics Students: Some Answers from Psychology," Proceedings of the 2003 Human Computer Interface International Conference, Crete, Greece, July, 2003.

Impact and Evaluation

We believe that the impact of this work could be significant. At present, it is extremely difficult for lecturers and students to communicate mathematical material. When these tools have been developed, it will prove possible for both the lecturer and the student to prepare mathematics using a preferred method and

exchange them freely.

User testing will be performed on individual components of the system as well as the fully designed system. This evaluation will occur both concurrently with the design of the system components as well as in a final phase to evaluate effectiveness of the

system as a whole.

The evaluation process will be based on a set of quantitative and qualitative metrics. Quantitative metrics will measure the number of errors present in the translated documents and the speed and

reliability of recognition of complex documents. Qualitative metrics will include perceived usability, user-friendliness, and reliability of the software.

Dissemination and Sustainability

The traditional forms of dissemination will be employed, E.g.:

<>1. Publications of research and educational results in international technical forums. <>2. Development of technical and educational reports,
distributed through department and project-specific web sites.

3. Creation of web sites describing our research program, outlining the expertise of the participants and their accomplishments (e.g., publications).

<>4. Technology transfer, e.g., deployment of research software prototypes developed by the investigators and made available on our web site. <>

Additional dissemination efforts will include preparation of written reports to the sponsoring agency and local presentations (e.g., workshops to increase public awareness about the project and school visits).

In the longer term, we plan to collaborate with partner institutions in other countries, with a view to obtaining funding from organisations such as the National Science Foundation or the European Union. Indeed, such inter-institutional collaboration is already in place and has proven to be most effective.

Detailed Budget

The budgetary requirements for this project are simple.

1. A budget of ?9,000 is being requested to fund two Research Assistants. This ammount is based on an annual salary of ?21,713 and includes the 10.75% contributions which, under law, employers must pay.

2. ?6,000 is being sought to procure a Refreshable Braille Display and Input Device which will be used to develop the software outlined above.

The reasons that these costs do not fall under the remit of the School of Computing is that they are all specifically related to this project, and not in any way school-related.

Support from Head of School/Unit

I have spoken to the head of the School of Computing (Prof. Michael Ryan) and he is fully supportive of this proposal. He believes it is a novel and worthwhile solution, and he hopes will facilitate a more inclusive education for blind students.