4:00 - 4:05 p.m. - Announcements (5 minutes)
Robert Wolpert – Professor Emeritus Statistical Science Department chairing for David MacAlpine, welcomed everyone to ITAC. We are continuing with the summer series of pulling information together on faculty and research needs with the 5th and second-to-last chat with various academic areas at Duke. Today is the Humanities turn – we have a team of 5 people with 3 represented today from Art & Visual Studies, Romance Studies, we are missing History & Classical Studies.
4:05 - 4:50 p.m. - Digital Environments (Humanities) Research and IT Support, Augustus Wendell, Edward Triplett, Phillip Stern, Annette Joseph-Gabriel, Josh Sosin (30-minute presentation, 15-minute discussion)
What it is: Faculty representatives of the Humanities (Digital Environments) will be joining us to present upon their research/academic efforts, discuss the role IT currently plays in support of their work, and identify and review some of the areas for growth and additional opportunity between the Humanities and IT support.
Why it’s relevant: In an effort to learn about the overall character of research and research IT support throughout the University, as well as to explore commonalities between the needs of individual domains and Duke as a whole, ITAC will be hosting a series of presentations/discussions over the course of the summer semester with key researchers and their colleagues. These discussions will aim to distinguish the most prevalent services for which IT need to aim to provide institutional level support versus those that are surely essential for certain research but are not pervasively used, and so may be better supported from the school/institute/department/lab level. Ultimately, OIT is seeking to open better lines of dialogue with the major research efforts at Duke, to learn how to better support our researchers, overcome any gaps in the current system, and collaborate to identify new ways to assist in elevating Duke's Research Community as a whole.
Augustus Wendell, Assistant Professor of the Practice art, Art History and Visual Studies described his research as two pronged:
- It deals with computational methods for historical spatial research 3D modeling with embeded scholarly annotation into 3D modelling
- It looks for quantitative insights using agent-based modeling or even some machine vision and computer vision learning
I have a number of projects that illustrate some of where my work comes into play using visual art such as:
- neural networks
- dealing with VR, Augmented Reality, Extended Reality,
- Digital Imaging -2D, 3D, 4D,
- Computational Methods,
- Game engines
- User Interface and User Experience Design
- AI Methods for Arts and Humanities
Some of our software is windows specific (Classroom 6 of the Link for instruction) and relies on the MPS (OIT’s Multimedia Project Studio) for support for independent studies with students or collaborative research with other faculty members. We also have specialized labs in Smith, which do a good job of supporting our research endeavors such as the Extended Reality (XR) lab and in particular the DAHVC (The Digital Art History and Visual Culture Lab). These are our workspaces where we work with students and we work individually. We have a certain amount of autonomy in setting up those machines and maintaining them, which is really very helpful when we're working in a fast-paced research environment.
We also have some onsite support for instance Hannah Jacobs does a huge amount of work with us and helping manage the digital art history and visual culture lab where she'll work with us to develop the image for the computers every year and help make sure that the faculty members and student teams that are creating work there has have input into that process so that we're well equipped for the upcoming year.
We have a challenge with students wanting to work on their laptops which, in the computational media realm, are often underpowered for the work that's needed. This is a very difficult dialogue to have with students and comes back to this issue of where do they work and what's the studio culture when we're moving very quickly through commercial software that's very intensive. We're working now to explore VM options.
Web Based Technologies:
We use a number of web based technologies and are interested in exploring whether these can be supported internally at Duke instead of the current model where we go outside of Duke. This actually works quite well because the vendors we're working with are well equipped to deal with our needs and the specialized supporting associated. Reinforcing that not everything needs to be centered in at Duke such as when we need to support particular / niche markets like digital humanities web libraries, etc.
Machine Learning Resources:
Augustus Wendell: I also am starting to brush up against machine learning resources needed in classes. This is where we need a GPU enabled lab for courses that require that sort of horsepower and capacity.
Sometimes the timeline and deployment into fully managed labs is challenging and it's totally understood that there is a long timeline for that. This gets harder when revisions to software is happening relatively quickly and we may not have 9-12 months to prepare for the impact of those changes in our VR environments. While the MPS lab is well equipped to support the authoring, I think we're headed in a direction where we need to think about where we can stage and showcase the environments that interact with these.
Robert Wolpert: Then let's open this up for questions from audience I’ll sneak one in at the start. The state of technology in AR (Augmented Reality) and VR is moving very fast and new devices that have been announced by half a dozen different companies from Google to Apple to Oculus. Is that going to affect what you do, or is that just going to make it a little bit easier to do get high resolution 3D pictures for people.
Augustus Wendell: I think we will upgrade. When there are opportunities that are important from the market, we will lobby to upgrade our systems. One person has been quite helpful in this is Dave Zelinski who has a background in this area, and he helps keep an eye on the hardware and software setup, the headsets that can be distributed for projects, and other areas. So he's a good kind of first line of defense of what's coming up in the marketplace and when it's appropriate to be moving forward. Every year discussions occur around what's emerging and do we have adequate hardware.
Robert Wolpert: Is it weird that these things cost thousands of dollars and the technology is evolving fast so is there a worry that we will have to hemorrhage money to try to keep up?
Augustus Wendell: Yeah, so I think my perception is we've done a good job of not trying to keep up with everything like we're waiting and watching and we wait for a few cycles before we kick up and we're smart we're not just chasing the market.
Victoria Szabo: To what extent do you feel like we can create assets and then be agnostic in terms of the display technologies or how can we think about creating workflows that enable us to do that.
Edward Triplett: The concept of interoperability is really critical as is the idea of blending environments and experiences is so essentially—the concept of the metaverse is not going away. Being able to have our work exist and have a concept of where it exists in the real world, as well as in our 4k screens is one thing that we need to be aware of. But we're not there yet, and we have to be patient and embrace open source technologies and capabilities, like image interoperability frameworks.
Tracy Futhey: I wondered what are your observation about the extent to which some of the techniques and processes you all are building are getting to the point where they become more scalable… the reason I asked it is that when I saw the map you created it begs the question of whether we could have an efficient way to take Duke’s many maps and convert them into the 3D version along with all the all the augmented data that you have. Is that even imaginable, or are we talking decades off and needing hundreds of people to help do a process like that?
Augustus Wendell: Currently, on this project it's hand labor in the tracing which suggests that it would not be feasible, though I had a student in my AI for Arts and Humanities class who was testing an automated training and neural network to identify footprints in sand born maps. I think that's one of the beauties of our courses being intersection with students coming from computational backgrounds
As far as the developing systems that are not tied to one piece of hardware, having someone like Dave Z (an onsite IT staff person, who is part of the infrastructure support for these initiatives) is incredibly helpful. By having some of the staff members involved it really helps with accountability and keeping an eye on best practices moving forward, and what makes sense across the institution, not just in our department or our research work.
Robert Wolpert: A question about hardware and VMs and students using Macs… I hope that they can use those to connect to a VM?
Edward Triplett: Yeah that's a very good question… if we are going to use remote machines, we're using an application called SplashTop for homework sessions. This lets students remote into PCs that we actually have control of in our labs (Classrooms 6).
Robert Wolpert: So, is this an area that we should consider investing in and pushing forward more?
Edward Triplett: If it were me, yes, I but I’m certainly biased in this area, I think.
The difference between students remoting into desktops in Classroom 6 and them having their own their own VM Is really different in terms of their habits of work regarding saving things that go on the virtual machine.
Augustus Wendell: I also put in the chat that arising from some of our earlier group discussions this summer, we're working with the VM group and they were very enthusiastic about testing scalability of VM for these more intensive applications so we've given them two case studies of courses that would typically not work well at a low level VM but we're going to be testing them out.
Robert Wolpert: Yeah that sounds like a pilot study would be helpful yeah.
Robert Wolpert: On another question the visualization is also important in STEM, in chemistry and statistics and in a number of other fields. Is there any synergy or any overlap that we're not exploiting yet between the what the humanities use of visualization and the sciences.
Augustus Wendell: I think that it would be smart to bring those people together, I mean I have my perception which is that our bias towards using more commercial software to solve these issues would mean that there isn't much to share, but I think it would be sensible to have some dialogues about resources and about understanding how we could work together, where we complement each other.
Edward Triplett: Oh, just to answer that quickly, I think the environmental science in particular has intersected, such as students who have come over from the Nicholas school and bring a lot of interesting ideas and where our paths cross quite a lot
Houdini in particular it's actually a very, very good simulator of physics for making things blow up, but in the world of physics it's actually quite applicable in that way and I’ve seen quite a lot of work in that in that direction has been interesting.
Robert Wolpert: This might be an area where we can try to get some people in the room to talk to each other is 3D printing and imagining that you might actually build a scale model.
Edward Triplett: My students are always interested and when they finish their final projects, I get I’ll get a little model on my desk.
[responding to a question in chat]
Edward Triplett: So, physics, particle simulation and fluid dynamics this kind of thing it's really interesting to the people in the visual effects world and the kind of underlying code that makes it look realistic.
Steffen A Bass: All right, if the key is to look realistic, yes, physics as essentially. To have something photo classical trajectories or the use of gravity so that it looks like we expect it to look in the real world it, it has very little to do with when a physicist speaks about particles relations of fluid dynamics, because that that that that happens on a different scale.
Edward Triplett: I agree completely; there is one thing I really like about this particular software Houdini— it's really easy to get under the hood.
Robert Wolpert: Steffan is an expert in what happens when heavy ions come at each other and almost the speed of light.
Steffen A Bass: We do really explosions with relativistic dynamics and I can assure you Houdini can’t do that.
Edward Triplett: One last thing is about the physical desktop. The Mac laptop culture and what happens with you when you have that Mac laptop. It sorts of bleeds into other aspects of teaching and research and that students don't quite understand that when you're doing graphics work you, above everything else you need a separate monitor and you need a mouse, and you need to kind of change your posture from the footprint of the Mac laptop. If we don't have desktops that are available for students then what they're doing is not going to match what they will do when they get out into the real world, or if they're working remotely their own desktop computer. A desktop computer is just the right tool for the job, otherwise it really is an uphill battle.
I have a student that will buy that will wait to buy a mouse until the second to last week of class. Every single semester, and then they finally realize oh wow I can do everything so much better so it's just it's just an aspect to this, I think we don't really think about a lot.
Also, I have had experiences where hardware issues have caused students to drop my GIS course. I am a huge advocate for GIS and other spatial software, so it is disappointing when I see a graduate student get discouraged and move away from mapping or modeling for their thesis project because of access issues. I also think physical labs are so important because this is where students teach each other and learn by looking over each other’s shoulders. Dedicated computing spaces are critical for building communities in our project-based courses and projects – especially Bass Connections, Code+, Data+, Story+ etc.
Augustus Wendell: I think that came up a lot in our earlier discussion this summer was where students work on this sort of intense semester long push with very aggressive timelines and the one chance to learn this and that by being on their own and fractured it's challenging and that there it's hard to find those spaces for the specialized areas, we have our classroom but beyond that it's labs that are very small that have a few seats and they're well equipped, but where where's the space that the students can congregate learn together work on appropriate hardware.
Robert Wolpert: Do the students come in familiar with gaming hardware
Augustus Wendell: I’m not familiar with that, but I think there is a very strong culture of the Mac laptop here; it seems across the board, they show up with their lightweight laptop.
Robert Wolpert: Really interesting presentation and thought provoking to figure out what how we can better support the current and emerging needs that you have.
Augustus Wendell: Thank you Tracy, Thank you.
4:50pm - 5:30 p.m. - 2022 Summer Code+ Presentations, Jen Vizas, Isabel Valls, Duke Code+ Student Groups (40-minute presentation(s) and following discussion)
What it is: Two student teams from the Code+ summer program (codeplus.duke.edu)
- Using ML/NLP to Identify Relationships and Similarities in Grant Proposal Texts
- Duke FixIt: An Issue Reporting App for Duke
Each presentation will be followed by a few minutes for Q&A.
Why it’s relevant: In its fifth year, Code+ has returned to an in-person program that has embraced the challenge of creating unique and inspired systems to further the experience for Duke Faculty, Staff and Students. Led by OIT staff, student teams are acquiring technical and soft skills, expanding their experience outside the classroom and preparing them for future tech internships and careers.
Robert Wolpert: Summer Code+ presentations with Jen and Isabel.
Jen Vizas, Duke-OIT: Code+ is a 10 week project a summer coding experience for undergraduate students that’s part of the larger suite of co-curricular programs that we affectionately referred to as the +Programs, which includes data+ CS+ Code+. This year we were thrilled to have all programs in person and co located over and Gross Hall. This year we had 45 students on a project team, so it was really difficult to choose the projects to bring here today.
The team that will report is focused on: “Using Machine Learning and Natural Language Processing to Identify Relationships and Similarities in Grant Proposals” and this project team had Mark McCahill and Katie Kilroy as the project leads and mentors.
Presentation by the team of Students:
Rodrigo Bassi Guerreiro, Rose DiPietro, Isabelle Xiong, Prince Ahmed, Quan Doan,
Tracy Futhey: Thank you for your work on this and your various presentations to different stakeholder and interest group bodies.
Rodrigo Bassi Guerreiro: Well, thank you so much for listening through all of them again.
Brandon M. Lê: This is a great presentation and as someone who writes grants myself I’m very curious about the implementation of this. I was wondering about one particular section of grants that's present in every application—the references section I’m wondering if you picked up any instances of grants being matched more closely with each other just because they cite a lot of the same sources and if not, whether you actually look into matching grants based on you know the similarity of the references?
Rodrigo Bassi Guerreiro: Yeah. So, I guess, I can answer that we did indirectly look into that one of the natural language processing models that we use called vector is trained specifically on. research and academic material in general, and it does exactly that comparison like building relationships between the different references, but I think as Prince mentioned our process for determining which model, we were ultimately going to use was very empirical, which is when based off of what returned the best results with the examples that we had. For some reason that one just didn't work, as well as the one that we ended up choosing. But yeah, we did, that is a very good point and, hopefully, like in the future that could even be incorporated, depending on how this project ends up going on.
Emma Fleischman: Yeah, I was wondering about the user input sliding scale. Based on the sentence pairs, does that go back into the model and teach it to do better if it if it was a poor match or what is the purpose of that.
Prince Ahmed: So, the question is, is the user feedback incorporated in the model?
The models are kind of black boxes and they don't get changed or updated as the models goes on, but with the information, we can see how well the model is performing in the future and we could even deploy different models and collect the user feedback, but for now we'll just anticipating humans can then go and see how the model is performing. So it's more like a snapshot how its performing rather than a continuously updated model.
Victoria Szabo: Thanks so much for sharing what you're doing I was thinking about the question of visualization and different strategies for engaging with your data, do you have ideas for further development in that sort of area?
Prince Ahmed: I’m not sure what visualization you're referring to. Our product is mostly to do with finding similar documents, so we don't necessarily visualize those document. We just process the documents and we try to find very similar ones.
Victoria Szabo: I was thinking about exploration of the results. So that's a different way of thinking about the user and engaging with what you've discovered.
Prince Ahmed: All right, yes, so for the user interface We work closely with the office that it's going to use this product, so they already have a similar product that does more like low level matching and we they were very interested in the sentence by sentence so that's why we display those so.
Prince Ahmed: We had a series of interviews with them and we just asked specifically what they wanted to visualize and we just visualize those but thank you.
Jen Vizas, Duke-OIT: We have one more project team.
Esha Kapoor: Hi everyone Thank you so much for coming. Our Code+ team worked on creating a centralized issue reporting web app for students faculty and staff called Duke FixIT I'm Esha, and this is our team and our wonderful team lead Sherry Tips.
Esha Kapoor: First, will explain why we decided to create fix it then we'll show you a DEMO of our product briefly go over the technologies that we use the challenges that we faced our future plans and then I’ll answer any questions at the end now I’ll pass it on to Emma to talk about the problem.
Project Team is comprised of:
Esha Kapoor, Emma Fleischman, Jennifer Xu, Haojin Li,
Robert Wolpert: To our presenters, thank you very much, this is quite remarkable, we have just a couple of minutes for questions people have anything you'd like to contribute.
Jen Vizas, Duke-OIT: Excellent job teams.
Tracy Futhey: Yeah thanks to everybody for a fantastic job, once again, not only with the work you've developed, but also with your presentation.
Jen Vizas, Duke-OIT: And I’ve mentioned previously at the end of our Code+ program what we do is we look at the projects their status and we talked to the team leads and the stakeholders to determine which projects will continue forward, which has legs and which ones the stakeholders want to pursue.
Tracy Futhey: Again, another commercial for tomorrow, anybody who wants to come to the +Program poster sessions, they are being held in Gross Hall tomorrow afternoon from 2-4 is welcome and you can hear more about these projects and 40 or 50 other projects between Data+ Code+ and CS+ that will be available tomorrow,