This is a list of links and bibliographic references related to the evaluation of digital work in the arts and humanities. This section was edited by Ronnie Apter.
Guidelines
Guidelines of Academic Organizations
“This document is … as a description of circumstances, standards, and practices within the field. Frequently, colleagues and administrators are unaware of many critical issues in this rapidly developing area, and therefore initial hiring interviews and subsequent performance reviews can be difficult for both faculty and administrators. This document is presented as a set of guidelines to assist in faculty hiring, promotion and tenure, workload, compensation, funding, and support in this field, and to provide information about faculty working in this area that could be used in making accurate and comprehensive evaluations.” Advise is given to both evaluators and candidates.
http://www.ncte.org/cccc/resources/positions/promotionandtenure
http://www.ncte.org/cccc/resources/positions/digitalenvironments
http://www.ncte.org/cccc/committees/7cs/tenurepromotioncases
Since 1987, the CCCC has been providing teachers, scholars, and administrators with guidelines for engaging in and evaluating digital academic work. The earliest set of guidelines, Scholarship in Composition: Guidelines for Faculty, Deans, and Department Chairs (1987), focuses on scholarship of application, discovery, integration, and teaching in composition research. The CCCC Promotion and Tenure Guidelines for Work with Technology (1998 consists of 3 parts: “general statements about technology and its potential impact on the promotion and tenure review processes, specific guidelines for promotion and tenure committees; and specific guidelines for candidates for promotion and tenure.” Even more recently, the CCCC issued a Position Statement on Teaching, Learning, and Assessing Writing in Digital Environments (2004). This statement puts contemporary teaching and research praxes in context, explaining: “The focus of writing instruction is expanding: the curriculum of composition is widening to include not one but two literacies: a literacy of print and a literacy of the screen. In addition, work in one medium is used to enhance learning in the other. As we refine current practices and invent new ones for digital literacy, we need to assure that principles of good practice governing these new activities are clearly articulated.”
Readers of this Wiki may be especially interested in the fourth link: Tenure and Promotion Cases for Composition Faculty Who Work with Technology. This resources is sponsored by the CCCC Committee on Computers in Composition (the 7 Cs). It contains five fictional tenure and promotion cases of composition faculty who work with computer technology. It also includes comments from real college and university administrators (e.g., department chairs, deans, and personnel committee chairs), who discuss how particular cases would be evaluated at their schools.
Like other such guidelines, this report addresses the concerns of peer review, but it also address the problems of copyright ownership, difficulties of controlling change in and preserving digital texts (both because of their fluidity and because of technological change, and version control.
“This document has been drafted … (i) to establish a clearer understanding for departments, institutions, professional associations and decision-making bodies of the range of activities represented in the field, and (ii) to provide an organized and consistent perspective on the rubrics under which these activities should be evaluated. Assessment and academic recognition of work in CALL presents difficulties not only because of the rapid evolution of the field but also because of the emergence of new theoretical and methodological paradigms. CALL work can be categorised as research, development, and practice.” The document goes on to define those terms.
“[These]…guidelines, which deal with the hiring, reappointment, tenure, and promotion processes, are designed to help departments and faculty members European Association for Computer-Assisted Language Learning implement effective evaluation procedures.… Institutions and, when appropriate, departments should develop their own written guidelines….Written guidelines must provide clear directions for appointment, reappointment, merit increases, tenure, and promotion and should take into consideration the growing number of resources for evaluating digital scholarship.” Adivce is given to both evaluators dn candidates. Among the recommendations is the desideratum for committees to review work in the medium in which it was created.”
This report examines current standards and emerging trends in requirements for tenure and promotion in English and foreign language departments in colleges and universities throughout the United States. It recommends that the profession “develop a more capacious conception of scholarship” and that “departments and institutions should recognize the legitimacy of scholarship produced in new media, whether by individuals or in collaboration, and create procedures for evaluating these forms of scholarship.”
This document was prepared by representatives of the UC Language Consortium Steering Committee (http://uccllt.ucdavis.edu), which includes language faculty from each campus of the UC system, and approved on April 24, 2009. Its purpose is to provide the academic P&T committees with background information to assist them in evaluating applied linguists working in language, literature, and culture departments whose research deals with language pedagogy, CALL research, second language acquisition, applied linguistics or linguistics. As a response to departments seeking guidance in this area, we offer the attached white paper on “Considerations for Evaluation of Applied Linguists in Language, Literature, and Culture Departments in the UC System,” which includes a list of the most notable journals in this field.
Guidelines of Specific Departments or Academic Units
“This document strives to provide a resource which outlines criteria for evaluating dossiers in this scholarly area.”
This “argument for redefining promotion and tenure criteria for faculty in new media departments of today’s universities” suggests that ” Invitations to publish in edited electronic journals or printed magazines and books should be recognized as the kind of peer influence that in other fields would be signaled by acceptance in peer-reviewed journals” and states that “The 2003 National Academies study concludes that conferences on new media, both face-to-face and virtual, offer a more useful and in some cases more prestigious venue for exposition than academic journals.”
See also the Addendum from the New Media Department defining criteria: http://newmedia.umaine.edu/interarchive/promotion_tenure_redefinitions.html
This university does not specifically address evaluation of digital publications, but does have a criterion for evaluation of academic research which reads: “Computer-based activities are considered a part of contemporary scholarship in the humanities and may be assessed in a manner similar to that of print publications (including refereeing, weighing of collaborative efforts, originality, contribution to discipline etc.”
See also the PDF documents: http://english.uvic.ca/documents/scholeval-1.16.09.pdf>and http://web.uvic.ca/humanities/documents/ReportonAcademicComputingRecognition.pdf
- The University of Saskatchewan, Department of English:
The English Department has new guidelines for evaluating Digital Projects.
Organizations Interested in Digital Humanities
Among its many publications, the ADHO includes “The NINCH Guide to Good Practice in the Digital Representation and Management of Cultural Heritage Materials”: http://www.nyu.edu/its/humanities//ninchguide/
“The Text Encoding Initiative (TEI) is a consortium which collectively develops and maintains a standard for the representation of texts in digital form. Its chief deliverable is a set of Guidelines which specify encoding methods for machine-readable texts, chiefly in the humanities, social sciences and linguistics. Since 1994, the TEI Guidelines have been widely used by libraries, museums, publishers, and individual scholars to present texts for online research, teaching, and preservation.”
CALICO “is a professional organization that serves a membership involved in both education and technology.… [It] is a recognized international clearinghouse and leader in computer assisted learning and instruction. It is a premier global association dedicated to computer-assisted language learning (CALL)….CALICO now includes foreign language educators, programmers, technicians, web page designers, CALL developers, CALL practitioners, novice CALL users, second language acquisition researchers–anyone interested in exploring the use of technology for foreign language teaching and learning.”
The site has information on how to join, CALICO publications, both book series and journals, and the CALICO Conference.
TAPoR is a gateway to tools for sophisticated analysis and retrieval, along with representative texts for experimentation. The tools of TAPoR can be used to manage electronic texts, experiment with online text tools, learn about digital textuality.” Actual tools are online at this site for experimentation and for teaching techniques. Geoffrey Rockwell explains, “Much of the evidence used by humanists is in textual form…. The computer does not replace human interpretation, it enhances it. The concordance is an example of a research aide that predates computing. The concordance, like the index, allows the interpreter to find passages that share a common word about which you are asking. Unlike the index, the concordance is a presentation of the passages that “concord” together for reflection. Thus a Key Word In Context display shows one line of text with the key word searched for in the middle so that the reader can see if there are patterns in the way the word is used. Most text analysis tools build on the concordance. They break down the text (analysis) and then represent passages in new arrangements (synthesis) that aide the questioner list graphs of the distribution of a word. Text-analysis tools break a text down into smaller units like words, sentences, and passages, and then gather these units into new views on the text that aid interpretation.”
TADA has a wiki with materials on tool development for people interested in connecting with others who do development as part of their research practice. TADA is also developing a model for competition exchange that can provide an alternative to review for scholarly tools. See http://tada.mcmaster.ca/trex/>.
Journals of Interest
CALL-EJ Online is a international refereed professional journal on Computer Assisted Language Learning and related fields published semiannually.
“The CALL Review (CALL = Computer Assisted Language Learning) is the newsletter of the IATEFL Learning Technologies (formerly Computer) SIG. It is published three times a year, in spring, summer and autumn. Articles should be something between 800 and 3000 words (max). Papers should be sent as email attachments in any popular word processor format or as a PDF file. Any graphics should be sent as separate files (.jpg, .tif or .bmp).… The CALL Review is only available to members of IATEFL who have elected to belong to the Learning Technology (formerly Computer) Special Interest Group.”
Currently, the CCC Online Archive is a research tool and digital supplement to the journal College Composition and Communication. In the main the CCC Online Archive offers site visitors an extra point of access to the print journal along with various options for searching journal content and surveying (as well as generating) metadata.
- Journal of Asynchronous Learning Networks (JALN); ISSN 1092-8235 (online) – ISSN 1939-5256 (print)
“The aim of the Journal of Asynchronous Learning Networks is to describe original work in asynchronous learning networks (ALN), including experimental results. Our mission is to provide practitioners in online education with knowledge about the very best research in online learning. Papers emphasizing results, backed by data are the norm. Occasionally, papers reviewing broad areas are published, including critical reviews of thematic areas. Papers useful to administrators are welcome. Entire issues are published from time-to-time around single topic or disciplinary areas. The Journal adheres to traditional standards of double-blind peer review, and authors are encouraged to provide quantitative data; currently JALN’s acceptance rate is 25%. The original objective of the Journal was to establish ALN as a field by publishing articles from authoritative and reliable sources. The Journal is now a major resource for knowledge about online learning.”
“MERLOT [Multimedia Educational Resource for Leaning and Online Teaching] is a free and open resource designed primarily for faculty and students of higher education. The MERLOT Journal of Online Learning and Teaching (JOLT) is a peer-reviewed, open access, online publication addressing the scholarly use of multimedia resources in online education. JOLT is published quarterly in March, June, September, and December. The objectives of JOLT are to enable faculty to use technology effectively in online teaching and learning by learning from a community of researchers and scholars; enable academic programs to design and deploy academic technology to optimize online teaching and learning; build a community around the research and scholarly use of multimedia educational resources for online teaching and learning. JOLT welcomes papers on all aspects of the use of online multimedia educational resources for online learning and teaching.” Subscriptions to JOLT are free.
Founded in 1996, KAIROS is a refereed professional journal for teachers and scholars working in Rhetoric and Composition and related fields. As the journal’s website reports: “Kairos is a refereed open-access online journal exploring the intersections of rhetoric, technology, and pedagogy. The journal reaches a wide audience—currently 45,000 readers per month—hailing from Ascension Island to Zimbabwe (and from every top-level domain country code in between); our international readership typically runs about 4,000 readers per month. Kairos publishes bi-annually, in August and January, with occasion special issues in May. Our current acceptance rate for published articles is approximately 10%.”
- ReCALL; ISSN 0958-3440
“ReCALL is a fully-refereed journal published by Cambridge University Press on behalf of EUROCALL. It is issued three times a year in January, May and September and is also available online to subscribers. The May issue normally contains selected papers from the previous year’s EUROCALL conference. The journal contains articles relating to theoretical debate on language learning strategies and their influence on practical courseware design and integration, as well as regular software reviews.” Back issues are online.
Vectors is anew international electronic journal that brings together visionary scholars with cutting edge designers and technologists to propose a thorough rethinking of the dynamic relationship of form to content in academic research , focusing on ways technology shapes, transforms and reconfigures social and cultural relations.” Vectors is a publicaton of the USC School of Cinema and Television.
Articles
“The evaluation of projects in humanities computing can be difficult. The work behind these projects, as they have become more technologically complex and more collaborative, can be difficult to evaluate even for those who follow recent trends in the field. To many chairs who began their careers well before the explosion of the Internet in 1990 and the subsequent proliferation of Web sites and hypermedia projects, such work might seem puzzling, even a challenge to traditional methods of scholarship. How can chairs with limited familiarity in humanities computing or even computers evaluate a digital project? How can they distinguish a scholarly digital project from more formulaic Web sites? How can colleagues make visible the work that has gone into creating a digital archive? In this article I outline some recommendations based partly on my experience developing and expanding my digital archive The World of Dante (www.worldofdante.org).”
Forums, Collaborations, and Other Resources
CompPile is a working, open-access reference tool that provides open access to published work in post-secondary composition and rhetoric, from the beginning of WWII to the end of the twentieth century. To date, it covers the period between 1939 and 1999, offering an on-line, keyworded, searchable inventory for researchers and teachers and anybody else interested. CompFAQs is a wiki collection of information relevant to teachers, scholars, and administrators in writing studies and related fields. Always in process, it currently includes sections on writing assessment, professional development, and workload.
“The Globalization and Autonomy Online Compendium [is] a collective publication by the team of leading Canadian and international scholars who are part of the SSHRCC Major Collaborative Research Initiative on Globalization and Autonomy. Using the Compendium, the team is making the results of their research available to a wide public audience. Team members have prepared a glossary of hundreds of short articles on relevant persons, places, organizations, events and key concepts and compiled an extensive searchable bibliographical database. They have written short summaries of their research that will be published in academic form in the 10-volume UBC Press “Globalization and Autonomy Series: Dialectical Relationships Shaping the Contemporary World.” Finally, the Compendium contains position papers and peer-reviewed research articles on globalization and autonomy issues.”
“Humanist is an international online seminar on humanities computing and the digital humanities. Its primary aim is to provide a forum for discussion of intellectual, scholarly, pedagogical, and social issues and for exchange of information among participants.”
“MERLOT is a leading edge, user-centered, searchable collection of peer reviewed and selected higher education, online learning materials, catalogued by registered members and a set of faculty development support services. MERLOT’s vision is to be a premiere online community where faculty, staff, and students from around the world share their learning materials and pedagogy.” Members can contribute learning materials and get the materials reviewed.
This combined database and social networking space is the first online, open-access resource for writing researchers. The site includes plans for studies, descriptions of current studies in progress, reports of previously completed studies, data of various kinds that others might use as part of a study (student writing, portfolios, etc.), and local- or program-specific materials that were produced for other purposes, such as assessment. Anyone currently involved in writing research of any kind is invited to file a report on his or her project. Anyone teaching writing research or thinking of conducting a large- or small-scale study will find resources and a community of co-researchers.
“Rotunda was created for the publication of original digital scholarship along with newly digitized critical and documentary editions in the humanities and social sciences. The collection combines the originality, intellectual rigor, and scholarly value of traditional peer-reviewed university press publishing with thoughtful technological innovation designed for scholars and students. Our publications are produced by the staff of the Electronic Imprint of the University of Virginia Press.
Techrhet is an active listserv for teachers, scholars, and other writing professionals interested in any aspect of computers, writing, and rhetoric.
The WAC (Writing Across the Curriculum) Clearninghouse partners with the International Network of Writing Across the Curriculum Programs to publish journals, books, and other resources for teachers and scholars who use writing in their courses. The site is open to all and includes publications such as Academic Writing, Across the Disciplines, Language and Learning Across the Disciplines, and The WAC Journal, as well as numerous digital book publications.
Relevant Blogs and Personal Webpages
- Wikipedia entries and Tenure
Examples of Collaborative Digital Humanities Projects, 1 June 2009