Showing posts with label gpt. Show all posts
Showing posts with label gpt. Show all posts

Thursday, August 4, 2022

Accidental Autosomething


Putting this here for posterity, of the time we accidentally figured out how to engineer happy accidents into our artificial intelligence research, and as a reminder that GPT, a program meant to understand language, has inadvertently kicked off the Singularity.

AI Is My Copilot
Apr 2022, WIRED Clive Thompson

By late 2020, developers had observed something unexpected about GPT-3. The AI wasn't just good at autocompleting sentences. It could also autocomplete computer code. 

[So it was discovered by accident.]

GPT was trained on those bazillion documents scraped off the web, a lot of them were pages on which nerds had posted their computer code. That mean the AI had learned patterns not just in English but also in HTML, Python, and countless other languages. ... 
[Then they went crazy] 
Zaremba let the code-writing AI use three times as much computer memory as GPT-3 got when analyzing text. 
[Remember that this is the same program that was withheld from the public out of fear that it would be too powerful to control if it got into the wrong hands.] ...Potential abuses of this program, such as "scaled abuse," where someone using the AI could author a cloud of Twitterbots to harass a politician or spread disinformation ...

[Then they put GPT on GitHub and let it rip.]

The new program was called Copilot, and was meant to help you write code, and developers developed their own "theory of mind" about how Copilot works, and they got better at communicating with it. [That's right; we're getting better at empathizing with computers.] "It's not something you're used to. It's not like a human theory of mind. It's like an alien artifact that came out of this massive optimization." -AI Guru Andrej Karpathy
On Intellectual Property and Fair Use: "Some weren't thrilled that code they had put on GitHub for other humans to use had been masticated by an AI and turned into a potentially lucrative product for its owner. They wondered about the legality too. Sure, they'd posted the code as open source, but does treating it as training data count as a fair use?"

Final Thoughts: "Copilot even seems to have picked up knowledge about specific fields. Maria Nattestad is a software engineer at Google and, on the side, the author of a popular app that makes eye-catching visualizations from bioinformatics data. She discovered that Copilot knows about DNA. ... 

Image credit: Megastructure by Beate Bachmann for Pixabay, 2020 [link]


Tuesday, March 29, 2022

Through the Uncanny Valley


AKA Mimetic Supremacy


IBM's AI debating system able to compete with expert human debaters
Mar 2021, phys.org

The IBM system, known simply as Project Debater, scans the internet for such arguments and uses them in ways that it has learned are convincing.

Project Debater was asked to convince a panel of viewers that telemedicine was a good idea. Most of those on the panel found that the AI system did indeed change their stance on the topic—a possible indication that AI systems may one day soon play a role in human debates such as those that occur on social media sites.

via: Noam Slonim et al. An autonomous debating system, Nature (2021). DOI: 10.1038/s41586-021-03215-w


SEIHAI - The hierarchical AI that won the NeurIPS-2020 MineRL competition
Dec 2021, phys.org

The winning algorithm was called "sample-efficient hierarchical AI" or SEIHAI, and it combines machine learning, human observation, and more traditional computation tasks. The key is to break down the overall task into subtasks, and then finding examples of humans doing the subtasks, and from that, finding the best way to do each subtask. They significantly outperformed their competitors.

via Huawei Noah's Ark Lab at Tianjin University and Tsinghua University, winners of the Neural Information Processing Systems (NeurIPS) annual conference: Hangyu Mao et al, SEIHAI: A sample-efficient hierarchical AI for the MineRL competition. arXiv:2111.08857v1 [cs.LG], arxiv.org/abs/2111.08857


Researchers demonstrate first human use of high-bandwidth wireless brain-computer interface
Apr 2021, phys.org

All Brains Ahead

Participants with tetraplegia have demonstrated use of an intracortical wireless BCI with an external wireless transmitter. The system is capable of transmitting brain signals at single-neuron resolution and in full broadband fidelity without physically tethering the user to a decoding system. This wireless system is functionally equivalent to the wired systems that have been the gold standard in BCI performance for years,

via BrainGate and Brown University: John D Simeral et al. Home Use of a Percutaneous Wireless Intracortical Brain-Computer Interface by Individuals With Tetraplegia, IEEE Transactions on Biomedical Engineering (2021). DOI: 10.1109/TBME.2021.3069119


Engineers are designing an autonomous robot that can open doors, find nearest electric outlet
Nov 2021, phys.org

With a headline like that, why bother reading the article?

via University of Cincinnati: Yufeng Sun et al, Force-Vision Sensor Fusion Improves Learning-Based Approach for Self-Closing Door Pulling, IEEE Access (2021). DOI: 10.1109/ACCESS.2021.3118594


'Deepfaking the mind' could improve brain-computer interfaces for people with disabilities
Nov 2021, phys.org

When I tried to use the EPOC headset many years ago, I was faced with an existential quandry. You're supposed to put on the headset, and control a simple video game with your thoughts. But in order to prepare your headset, you have to teach it the first and most important lesson, which is the difference between having a thought and not having a thought.

Your brain isn't a quiet place, there's electric fields bouncing around all over the place, all the time, and in order to separate the signal from the noise, your headset has to known what is considered non-thought, "default state" activity. What does your brain sound like when it's not giving a command. Once we have that default baseline state, we can detect when you're giving an active, intentional command. Later on, we can start to differentiate commands, but to begin with, we just need to know what a command looks like, and what it doesn't.

To do this on the superscale, we would need to give the headset a whole lot of baseline data, which would require lots of different people to all wear these headsets all day.

Now, it looks like there's a way to make synthetic brainwave data, using generative adversarial networks (GANs), and then feeding it to the program. What used to take 20 minutes now takes 1 minute.

via  University of Southern California: Shixian Wen et al, Rapid adaptation of brain–computer interfaces to new neuronal ensembles or participants via generative modelling, Nature Biomedical Engineering (2021). DOI: 10.1038/s41551-021-00811-z


Wireless network controls brain circuits remotely via the internet
Dec 2021, phys.org

Relax, nothing to worry about

A new study shows that researchers can remotely control the brain circuits of numerous animals simultaneously and independently through the internet. 

The wireless ecosystem only requires a mini-computer that can be purchased for under $45, which connects to the internet and communicates with wireless multifunctional brain probes or other types of conventional laboratory equipment using IoT control modules.

"They can remotely perform large-scale neuroscience experiments in animals deployed in multiple countries," said one of the lead authors, Dr. Raza Qazi, a researcher with KAIST and the University of Colorado, Boulder.

via The Korea Advanced Institute of Science and Technology (KAIST), Washington University in St. Louis, and the University of Colorado, Boulder: Raza Qazi et al, Scalable and modular wireless-network infrastructure for large-scale behavioural neuroscience, Nature Biomedical Engineering (2021). DOI: 10.1038/s41551-021-00814-w


AI-generated faces found more trustworthy than real faces: Researchers warn of 'deep fakes'
Feb 2022, phys.org

Mimetic Supremacy Achieved

"Our evaluation of the photo realism of AI-synthesized faces indicates that synthesis engines have passed through the uncanny valley and are capable of creating faces that are indistinguishable—and more trustworthy—than real faces."

via Lancaster University: Sophie J. Nightingale et al, AI-synthesized faces are indistinguishable from real faces and more trustworthy, Proceedings of the National Academy of Sciences (2022). DOI: 10.1073/pnas.2120481119


DeepMind AI rivals average human competitive coder
Feb 2022, BBC News

Almost there

After simulating 10 contests, with more than 5,000 participants, AI system AlphaCode has ranked in the top 54% of competitors. There was still work to do to bring it up to the same level as top performing humans, DeepMind said.


Monday, March 28, 2022

Watch Words Come To Life


Turning an analysis of Asimov's Foundation into art
Oct 2021, phys.org

Just a great example of cross-disciplinary data art and network science, via Datapolis and the Central European University in Budapest: Milán Janosov, Flóra Borsi, Asimov's foundation—turning a data story into an NFT artwork. arXiv:2109.15079v1 [physics.soc-ph], arxiv.org/abs/2109.15079

See also this thing which I don't understand called an NFT platform: https://foundation.app/@milanjanosov/~/92747

Some interesting bits noted in the article:

  • We found that among the most mentioned keywords there were three different planets.
  • Different planets play different roles in the book. That's why we started to look at the emotional arcs of these planets.
  • We extracted a series of sentences about each planet, and we used a happiness scoring algorithm to create their emotional arc.
  • The arc of Trantor was going down, and that coincided with the fact that both Trantor and the galactic were also falling during the series; while on the other hand, Terminus, which is the heart of the Foundation, slowly rises.

Then they take an 8,000-word network graph and run it in the chronological order of the book, with the planets linking the words associated with them, but rising and falling throughout the arc of the narrative. So now it's a video. And they made their own soundtrack.

Post Script:
How does the brain interpret computer languages?
Mar 2021, Ars Technica

Interestingly, code-solving activated parts of the multiple-demand network that are not activated when solving math problems. So the brain doesn’t tackle it as language or logic -- it appears to be its own thing.

via MIT, Tufts: Comprehension of computer code relies primarily on domain-general executive brain regions. Anna A Ivanova et al. eLife, 2020. DOI: 10.7554/eLife.58906 


Post Post Script:
In a neuroprosthetic first, ALS patient sends social media message via brain-computer interface
Jan 2022, phys.org

It's called the Stentrode Brain Computer Interface, developed by brain computer interface company Synchron.

The real news here is that it's blurred the line over what we call invasive neural implants, since it was snaked through his jugular vein. This allows us to get really good signals from your brain without having to drill a hole in your head. 

via Synchron Press Release, "First Tweet by Implanted BCI", Dec 2021