Showing posts with label neural networks. Show all posts
Showing posts with label neural networks. Show all posts

Friday, July 19, 2024

The Large Photon Destroyer


Light is the next electricity. It runs quantum computers, it blows GPU-based neural networks out of the water, and it makes computing parameters like speed, bandwidth, power, etc., perform on scales we do not understand. The way the internet changed society is the way light will change computing. 

Scientists compute with light inside hair-thin optical fiber
Jan 2024, phys.org

I'll bet it does

"We can encode a lot of information on a single particle of light. On its spatial structure, on its temporal structure, on its color. And if you can compute with all of those properties at once, that unlocks a massive amount of processing power."

via Heriot-Watt University in Edinburgh: Inverse design of high-dimensional quantum optical circuits in a complex medium, Nature Physics (2024). DOI: 10.1038/s41567-023-02319-6.

Somewhat related image credit: the quantum tornado machine for black hole research - Leonardo Solidoro for University of Nottingham - Mar 2024


Key innovation in photonic components could transform supercomputing technology
Feb 2024, phys.org

Trying to get to the point because I'll bet it's important:

Programmable photonic integrated circuits (PPICs) - The key to the advance has been to apply innovative concepts to the fabrication of the required silicon-based parts. Crucially, the manufacturing process can be used with conventional silicon wafer technology. This makes it compatible with the large-scale production of photonic chips essential to commercial applications.

At the heart of the new advance are tiny components that can interconvert optical, electronic, and mechanical changes to perform the variety of communication and mechanical functions needed by an integrated circuit.

They reduced the power consumption to femtowatt levels, which is over a million times an improvement compared to the previous state of the art.

In a move away from the dependence on temperature changes required by the dominant "thermo-optic" systems currently in use, these new components manipulate a feature of light waves called "phase" and control the coupling between different parallel waveguides, which guide and constrain the light. 

via Daegu Gyeongbuk Institute of Science and Technology and Korea Advanced Institute of Science and Technology: Dong Uk Kim et al, Programmable photonic arrays based on microelectromechanical elements with femtowatt-level standby power consumption, Nature Photonics (2023). DOI: 10.1038/s41566-023-01327-5


Neural networks made of light: Research team develops AI system in optical fibers
Feb 2024, phys.org

NVIDA called, wants deep hype back

"We utilize a single optical fiber to mimic the computational power of numerous neural networks"

via Leibniz-Institut für Photonische Technologien: Bennet Fischer et al, Neuromorphic Computing via Fission‐based Broadband Frequency Generation, Advanced Science (2023). DOI: 10.1002/advs.202303835

PENTATRAP device for measuring quantum states - MPIK - Apr 2024

All-light communication network bridges space, air and sea for seamless connectivity
Feb 2024, phys.org

"The all-light communication could be used in oceans and lakes, for example, where sensors gather ecological data and communicate with surface buoys. The data could then be sent wirelessly over the water surface or across long-distance transmission links between cities. The network can also connect to the internet via a modem, granting people who might be in a remote ocean location, for example, access to the backbone network for information sharing."

  • They used blue light for underwater communication because seawater has a reduced absorption window for blue-green light, allowing it to travel farther underwater compared to other wavelengths.
  • White LEDs are used to transmit information between objects, such as buoys or ships that are above water.
  • For connections with airborne devices such as drones, deep ultraviolet light is used. This provides solar-blind communication, which prevents interference from sunlight.
  • Finally, for point-to-point communication in free space, near-infrared laser diodes were applied because they emit directional light with high optical power. 

via Nanjing University and Suzhou Lighting Chip Monolithic Optoelectronics Technology Co: Linning Wang et al, All-light communication network for space-air-sea integrated interconnection, Optics Express (2024). DOI: 10.1364/OE.514930


Using sound waves for photonic machine learning: Study lays foundation for reconfigurable neuromorphic building blocks
Apr 2024, phys.org

I don't really understand this but it's important 

  • reconfigurable neuromorphic building blocks
  • photonic machine learning

The researchers use light to create temporary acoustic waves in an optical fiber. The sound waves generated in this way can for instance enable a recurrent functionality in a telecom optical fiber, which is essential to interpreting contextual information such as language.

The sound waves have a much longer transmission time than the optical information stream. Therefore, they remain in the optical fiber longer and can be linked to each subsequent processing step in turn. 

FYI - A traditional fully connected neural network on a computer faces difficulties capturing context because it requires access to memory. In order to overcome this challenge, neural networks have been equipped with recurrent operations that enable internal memory and are capable of capturing contextual information.

Optoacoustic REcurrent Operator (OREO) - harnesses the intrinsic properties of an optical waveguide without the need for an artificial reservoir or newly fabricated structures.

via Stiller Research Group at the Max Planck Institute for the Science of Light and the Englund Research Group at the Massachusetts Institute of Technology: Steven Becker, Dirk Englund, and Birgit Stiller, An optoacoustic field-programmable perceptron for recurrent neural networks, Nature Communications (2024). DOI: 10.1038/s41467-024-47053-6.


Internet can achieve quantum speed with light saved as sound
Apr 2024, phys.org

Same thing as above

A small drum can store data sent with light in its sonic vibrations, and then forward the data with new light sources when needed again.

via University of Copenhagen's Niels Bohr Institute: Mads Bjerregaard Kristensen et al, Long-lived and Efficient Optomechanical Memory for Light, Physical Review Letters (2024). DOI: 10.1103/PhysRevLett.132.100802

Posted by at No comments:
Labels: , , , , , , ,

Thursday, January 11, 2024

Dematerialization and the Race for Asynchronicity


'Swarmalators' better envision synchronized microbots
Mar 2023, phys.org

The researchers simplified their model to work with just four mathematical constants linked together to produce diverse emergent behaviors, such as aggregation, dispersion, vortices, traveling waves, and bouncing clusters.

The new model can mimic particles in nature that each operate at different natural frequencies, as some objects move slower and faster around a trajectory than others. The researchers also added chirality, or the ability for a particle to move in a circle, because many examples in nature, such as sperm, swim in circles and in vortices. And particles in the model exhibit local coupling, so they sense and respond only to their local neighbors.

At its core, the model combines swarming behaviors with synchronization in time. 
"Swarmalators"

via Cornell University: Steven Ceron et al, Diverse behaviors in non-uniform chiral and non-chiral swarmalators, Nature Communications (2023). DOI: 10.1038/s41467-023-36563-4



Drones navigate unseen environments with liquid neural networks
Apr 2023, phys.org

First, a retronym in the making:
"Inspired by the adaptable nature of organic brains, researchers have introduced..."

You start referring to regular human brains as "organic brains" once some other kind of brain becomes important enough to force a distinction. 

The liquid neural networks can continuously adapt to new data inputs to make reliable decisions in unknown domains like forests, urban landscapes, and environments with added noise, rotation, and occlusion.

The new class of machine-learning algorithms captures the causal structure of tasks from high-dimensional, unstructured data, such as pixel inputs from a drone-mounted camera to extract crucial aspects of a task and ignore irrelevant features.

"Our experiments demonstrate that we can effectively teach a drone to locate an object in a forest during summer, and then deploy the model in winter, with vastly different surroundings. These flexible algorithms could one day aid in decision-making based on data streams that change over time, such as medical diagnosis and autonomous driving applications."

Unlike traditional neural networks that only learn during the training phase, the liquid neural net's parameters can change over time, making them not only interpretable, but more resilient to unexpected or noisy data.

Note, these are not the liquid neural networks described by others, where the "liquid" part of the analogy, or neologism, is literally a fluid that transports neurotransmitters, like hormones in the bloodstream or even antibodies in the immune system, and this differs from the idea of a neural network as one made of electric circuits. 

via MIT's Computer Science and Artificial Intelligence Laboratory: Makram Chahine et al, Robust flight navigation out of distribution with liquid neural networks, Science Robotics (2023). DOI: 10.1126/scirobotics.adc8892


In sync brainwaves predict learning, study shows
Apr 2023, phys.org

Students whose brainwaves are more in sync with their classmates and teacher are likely to learn better than those lacking this "brain-to-brain synchrony"

The researchers found that as students were listening to the lecture, their brainwaves became in sync with one another. Moreover, the researchers observed such "brain-to-brain synchrony"—similar brain-activity patterns over time—between the students' brainwaves and when comparing students' brainwaves to the teacher's brainwaves.

via NYU: The Temporal Dynamics of Brain-to-Brain Synchrony Between Students and Teachers Predict Learning Outcomes, Psychological Science (2023). DOI: 10.1177/09567976231163872

Post Script: If you look at the thumbnail for this story, it shows a woman wearing what looks like the Emotiv EEG headset. I bought one of those ten years ago for my high school students to try out, so they could experience playing video games with their minds. Immediately I realized that my students with tight curls (like "black people hair" vs "white people hair") definitely did not get the same connection -- the headset reads brainwaves via electrical currents, and if the headset can't make contact with the scalp, it can't read the electricity. That pissed me off and I stopped using it. Maybe they fixed that problem since ten years ago; maybe they didn't. That's what makes me wonder how science can perpetuate systemic racism, even though science is supposed to be blind to these things. In fact, some might say that the sole purpose of the scientific method is to reduce the bias of your investigation to the smallest amount possible, thus revealing as much of the truth as possible. 

Image credit: AI Art - Multi Cassette Ghetto Blaster Robot Head - 2023

Swarming microrobots self-organize into diverse patterns
Jun 2023, phys.org

The microrobots in this case are 3D-printed polymer discs, each roughly the width of a human hair, that have been sputter-coated with a thin layer of a ferromagnetic material and set in a 1.5-centimeter-wide pool of water.

The researchers applied two orthogonal external oscillating magnetic fields and adjusted their amplitude and frequency, causing each microrobot to spin on its center axis and generate its own flows. This movement in turn produced a series of magnetic, hydrodynamic and capillary forces.

"By changing the global magnetic field, we can change the relative magnitudes of those forces, " Petersen said. "And that changes the overall behavior of the swarm."

But wait -- "The reason why we're always excited when the systems are capable of caging and expulsion is that you could, for example, drink a vial with little microrobots that are completely inert to your human body, have them cage and transport medicine, and then bring it to the right point in your body and release it," Petersen said. "It's not perfect manipulation of objects, but in the behaviors of these microscale systems we're starting to see a lot of parallels to more sophisticated robots despite their lack of computation, which is pretty exciting."

(Yes, drinking a glass of microbot swarms does sound like the ideal method of drug delivery, yes it does.)

Also, just a reminder: The Swarmalator is swarming oscillator model

via Cornell and Max Planck Institute for Intelligent Systems: Steven Ceron et al, Programmable self-organization of heterogeneous microrobot collectives, Proceedings of the National Academy of Sciences (2023). DOI: 10.1073/pnas.2221913120


COVID lockdown - Are high-income earners more resistant to returning to the office?
Aug 2023, phys.org

I'm just here because this is the first paper published by Northeastern's Network Science Institute program in London: Northeastern expanded its world leading Network Science Institute to the university's campus in London this summer (2023) in a move to establish a new European hub in the fast-growing research field of network science.

But I stayed for the word synchronicity: High-income workers have the leverage to negotiate more for remote work, Di Clemente says. "They are the ones that can actually change their synchronicity," he says, adding that part of that workforce "might never come back" to physical offices full time.

via Northeastern University: Clodomir Santana et al, COVID-19 is linked to changes in the time–space dimension of human mobility, Nature Human Behaviour (2023). DOI: 10.1038/s41562-023-01660-3


A system to keep cloud-based gamers in sync
Aug 2023, phys.org

Listen up, writers of interplanetary science fiction:

Their system, called Ekho, adds inaudible white noise sequences to the game audio streamed from the cloud server. Then it listens for those sequences in the audio recorded by the player's controller.

Ekho uses the mismatch between these noise sequences to continuously measure and compensate for the interstream delay.

via MIT and Microsoft: Ekho: Synchronizing Cloud Gaming Media Across Multiple Endpoints. Pouya Hamadanian, D Gallatin, M Alizadeh, K Chintalapudi

Image credit: TPUv3 Pod - Google Labs - 2018

Making sense of life's random rhythms: Team suggests universal framework for understanding 'oscillations'
Aug 2023, phys.org

Studying stochastic, random oscillations like the synchronized blinking of fireflies, the back-and-forth motion of a child's swing, slight variations in the the human heartbeat. "If your heart cells aren't synchronized, you die of atrial fibrillation," Thomas said. "But if your brain cells synchronize too much, you have Parkinson's disease, or epilepsy,

"We turned the problem of comparing oscillators into a linear algebra problem"

Most oscillations are irregular; a natural variation of 5-10% in the heartbeat is considered healthy. "In San Francisco, modern skyscrapers sway in the wind, buffeted by randomly shifting air currents—they're pushed slightly out of their vertical posture, but the mechanical properties of the structure pull them back. This combination of flexibility and resilience helps high-rise buildings survive shaking during earthquakes. You wouldn't think this process could be compared with brain waves, but our new formalism lets you compare them."

via Case Western: Alberto Pérez-Cervera et al, A universal description of stochastic oscillators, Proceedings of the National Academy of Sciences (2023). DOI: 10.1073/pnas.2303222120


Fireflies, brain cells, dancers: Synchronization research shows nature's perfect timing is all about connections
Sep 2023, phys.org

They figured out a way to predict the synchronization between coupled oscillators by the network structure that connects them, and have revealed the impact of patterns of network connections among small groups of nodes (motifs) on the whole of network synchronizability. Results implicate the prevalence of clustered structure such as feedforward and feedback loops as the most important factor in synchronizability.

"Clustered structure"

"We present an analytic technique to directly measure the relative synchronizability of noise-driven time-series processes on networks, in terms of the directed network structure, and reveal subtle differences between the motifs involved for discrete or continuous-time dynamics.  

via University of Sydney and Max Planck Institute for Mathematics in the Sciences in Leipzig: Joseph T. Lizier et al, Analytic relationship of relative synchronizability to network structure and motifs, Proceedings of the National Academy of Sciences (2023). DOI: 10.1073/pnas.2303332120


Adaptive optical neural network connects thousands of artificial neurons
Oct 2023, phys.org

A network consisting of almost 8,400 optical neurons made of waveguide-coupled phase-change material; the connection between two each of these neurons can indeed become stronger or weaker (synaptic plasticity), and that new connections can be formed, or existing ones eliminated (structural plasticity). 

These synapses were not hardware elements but were coded as a result of the properties of the optical pulses -- in other words, as a result of the respective wavelength and of the intensity of the optical pulse. This made it possible to integrate several thousand neurons on one single chip and connect them optically. 

(btw) The researchers tested the performance of the neural network by using an evolutionary algorithm to train it to distinguish between German and English texts. The recognition parameter they used was the number of vowels in the text. 

via Collaborative Research Center 1459 (Intelligent Matter) at University of Münster and Universities of Exeter and Oxford: Frank Brückerhoff-Plückelmann et al, Event-driven adaptive optical neural network, Science Advances (2023). DOI: 10.1126/sciadv.adi9127


Posted by at No comments:
Labels: , , , , , , ,

Fundamental Organizational Principles and the Networking of Science

 

Do higher-order interactions promote synchronization?
Apr 2023, phys.org

Researchers use networks to model the dynamics of coupled systems ranging from food webs to neurological processes. Those models originally focused on pairwise interactions, or behaviors that emerge from interactions between two entities. But in the last few years, network theorists have been asking, what about phenomena that involve three or more?


Network theorists call these phenomena "higher-order interactions." Now scientists show how the choice of network representation can influence the observed effects, focusing on the phenomenon of synchronization, which emerges in systems from circadian clocks to vascular networks.

They compared hypergraphs of "hyperedges" to connect three or more nodes, and simplicial complexes, more structured and using triangles to represent  connections. 

In the paper, Zhang and his colleagues reported that networks modeled with hypergraphs easily give rise to synchronization, while simplicial complexes tend to complicate the process due to their highly heterogeneous structure. That suggests choices in higher-order representations can influence the outcome, and Zhang suspects the results can be extended to other dynamical processes such as diffusion or contagion.

"Structural heterogeneity is important not just in synchronization, but is fundamental to most dynamic processes," he says. "Whether we model the system as a hypergraph or simplicial complex can drastically affect our conclusions."

via Santa Fe Institute: Yuanzhao Zhang et al, Higher-order interactions shape collective dynamics differently in hypergraphs and simplicial complexes, Nature Communications (2023). DOI: 10.1038/s41467-023-37190-9



Researchers investigate the veracity of 'six degrees of separation'
Jun 2023, phys.org

I don't think I understand why 6 and not another number; but it appears that the big deal here is that the mechanism behind 'why 6' is based on a cost-benefit algorithm.

The intriguing phenomenon, they show, is linked to another social experience we all know too well -- the struggle of cost vs. benefit in establishing new social ties.

"6 Degrees" is from Stanley Milgram at Harvard in 1967 who used the United States Postal System to perform experiments on and model our social network.

(This point in itself is interesting to consider, in light of having the access to the electronic communications network that is the internet, and which later proved these experiments on the scale of millions not hundreds, that we did have already such a pervasive, well-functioning network at hand, in the form of the United States Postal Service.)  

Milgram sent letters to random people, with instructions to try and make it back to one of his professor-friends somewhere else across the country. The experiment found that it only takes about six handshakes to bridge between two random people.

So what is the common denominator?

The objective of using a social network for the individual, is not simply to pursue a large number of connections, but to obtain the right connections, for example, seeking a junction that bridges between many pathways, and hence funnels much of the flow of information in the network.

But social capital does not come for free. It requires constant maintenance. A constant buzz driven by the ambition for social centrality.

"We discovered an amazing result: this process always ends with social paths centered around the number six. This is quite surprising." (Dark side reminder: "Indeed, within six infection cycles, a virus can cross the globe.")

via Bar-Ilan University as well as collaborators from Israel, Spain, Italy, Russia, Slovenia and Chile: I. Samoylenko et al, Why Are There Six Degrees of Separation in a Social Network?, Physical Review X (2023). DOI: 10.1103/PhysRevX.13.021032

Post Script: The book Bursts by Albert-László Barabási does a great job of looking at these kinds of social network effects, following the travels of the Where's George campaign of dollar bills through the US for example. He took the incipient revelations of network science (the 6 degrees rule) and gave it a temporal dimension -- he showed how our activities can be measured in short bursts followed by not much activity at all. It's not just the '2-dimensional' shape of the network, but the extra time dimension that defines the salient behavior of the social network.
Bursts: The Hidden Pattern Behind Everything We Do
Albert-László Barabási, 2010


Researchers identify mathematical rule behind the distribution of neurons in our brains
Aug 2023, phys.org

I don't see the word network science in here but it is --

Researchers have uncovered the ubiquitous lognormal distribution of neuron densities across and within cortical areas in the mammalian brain, suggesting a fundamental organizational principle.

via Human Brain Project, Forschungszentrum Jülich and the University of Cologne: Aitor Morales-Gregorio et al, Ubiquitous lognormal distribution of neuron densities in mammalian cerebral cortex, Cerebral Cortex (2023). DOI: 10.1093/cercor/bhad160



Posted by at No comments:
Labels: , , , , ,

Tuesday, April 19, 2022

The Social Behavior of Optical Quantum Gas


Liquid light shows social behaviour
Oct 2022, phys.org

Too many whats all in one place. I had to read this one carefully.

First of all, Bose-Einstein Condensates (BECs) have been a favorite over here at Network Address for a long time. It's one of those metaphysical-sounding things that doesn't behave how we expect. It's considered two-dimensional, a description used to organize lots of materials (like graphene, or twisted nanosandwiches) that behave so alien to our understanding of physics that they seem to be operating in another dimension.

Like other metamaterials, BECs also use super-something to describe their behavior, like superconductor, superinsulator, superfluid. They usually require absolute peace and quiet in order to do this magic condensation trick, which means it needs to be really cold, like absolute zero cold. But these scientists have figured out how to do it at room temperature, and that's is a pretty big deal.

In this case, the BEC is made of photons, hence "liquid light" -- they created a structure of microcavities and mirrors that condense photons in an optical medium of rhodamine dye and a thermo-responsive polymer, and turn them into a two-dimensional superfluid.

But wait, there's more -- when trying to explain the behavior of these super-photons, there is talk of the liquid "deciding" what to do, and of "social behavior". (Sociothermodynamics perhaps?)

I should mention that 1. the writer calls the photon fluid a liquid, but the scientists call it a gas, and 2. the writer quotes the scientists as using the term "social behavior", but that term is not in the paper itself, and I definitely don't understand this enough to get the analogy. (Although it may have something to do with "backreflection" like the backpropagating feedback loops characteristic of neural networks.)

via University of Twente, Netherlands: Mario Vretenar et al, Modified Bose-Einstein condensation in an optical quantum gas, Nature Communications (2021). DOI: 10.1038/s41467-021-26087-0

Image credit: Quantum Thing, Getty Images, 2021

Post Script:
Researchers guide a single ion through a Bose-Einstein condensate
Jan 2021, phys.org

via University of Stuttgart:  T. Dieterle et al. Transport of a Single Cold Ion Immersed in a Bose-Einstein Condensate, Physical Review Letters (2021). DOI: 10.1103/PhysRevLett.126.033401

Posted by at No comments:
Labels: , , , , , ,

Friday, March 25, 2022

Neuromimetics


Nobody uses the word neuromimetic, but that's what it is. 

Instead, we're calling these new computer chips neuromorphic, which you would think means that their design is shaped (morphed) like the brain. Except that's not what it means -- instead neuromorphic computer chips ----behave---- like the brain. I would call that mimetic. 

This post is a collection of news articles about the next generation in computing -- no, not GPUs, I know we're just getting familiar with those, but the next thing is already here. This next, next thing coming is a TPU (tensor processing unit), and it's got something to do with the memory being on the chip, so you don't need to go across the bus to an external memory. Sorry, not a computer scientist here, just trying to get the basic idea.

These new chips can create a new type of architecture that's really good at really large datasets, which we happen to have (digital content on track to equal half Earth's mass by 2245).

These chips act more like synapses, hence the term neuromorphic. I'll be drifting in and out of this topic specifically, since some of these are just regular old GPU-based neural networks, which as you would guess by their names, are a kind of neuromorphic hardware in themselves. Also, cerebral organoids. Less mimetic. They're made of actual brain tissue, but we can't talk about brain-like things without talking about those. 


New approach found for energy-efficient AI applications
Mar 2021, phys.org

Using not just spike activation and inhibition but the temporal pattern, that reduces the dimensions...

I get confused when someone talks about "artificial" neural network, because I thought the whole thing was an artificial brain to begin with. But alas --

"This low energy consumption is made possible by inter-neuronal communication by means of very simple electrical impulses, so-called spikes. The information is thereby encoded not only by the number of spikes, but also by their time-varying patterns. "You can think of it like Morse code. The pauses between the signals also transmit information," Maass explains.

via Graz University of Technology: C. Stoeckl and W. Maass. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence. (2021) DOI: 10.1038/s42256-021-00311-4


'Edge of chaos' opens pathway to artificial intelligence discoveries
Jul 2021, phys.org

An artificial network of nanowires can be tuned to respond in a brain-like way when electrically stimulated. Keeping the network of nanowires in a brain-like state "at the edge of chaos", it performed tasks at an optimal level. Electrical signals put through this network automatically find the best route for transmitting information. And this architecture allows the network to 'remember' previous pathways through the system.

This, they say, suggests the underlying nature of neural intelligence is physical, and their discovery opens an exciting avenue for the development of artificial intelligence.

via University of Sydney and Japan's National Institute for Material Science: Nature Communications (2021). DOI: 10.1038/s41467-021-24260-z

Tiny brains grown in 3D-printed bioreactor
Apr 2021, phys.org

Exactly what it sounds like.

via American Institute of Physics, MIT and the Indian Institute of Technology Madras: "A low-cost 3D printed microfluidic bioreactor and imaging chamber for live-organoid imaging" Biomicrofluidics (2021). aip.scitation.org/doi/10.1063/5.0041027


Team presents brain-inspired, highly scalable neuromorphic hardware
Aug 2021, phys.org

Good explanation in the writeup by KAIST :

Neuromorphic hardware has attracted a great deal of attention because of its artificial intelligence functions, but consuming ultra-low power of less than 20 watts by mimicking the human brain. To make neuromorphic hardware work, a neuron that generates a spike when integrating a certain signal, and a synapse remembering the connection between two neurons are necessary, just like the biological brain. However, since neurons and synapses constructed on digital or analog circuits occupy a large space, there is a limit in terms of hardware efficiency and costs. Since the human brain consists of about 1011 neurons and 1014 synapses, it is necessary to improve the hardware cost in order to apply it to mobile and IoT devices.

To solve the problem, the research team mimicked the behavior of biological neurons and synapses with a single transistor, and co-integrated them onto an 8-inch wafer. The manufactured neuromorphic transistors have the same structure as the transistors for memory and logic that are currently mass-produced. In addition, the neuromorphic transistors proved for the first time that they can be implemented with a "Janus structure' that functions as both neuron and synapse, just like coins have heads and tails.

via The Korea Advanced Institute of Science and Technology: Joon-Kyu Han et al, Cointegration of single-transistor neurons and synapses by nanoscale CMOS fabrication for highly scalable neuromorphic hardware, Science Advances (2021). DOI: 10.1126/sciadv.abg8836


Reappraisal of Moore's law through chip density
Aug 2021, phys.org

"DRAM chips as model organisms for the study of technological evolution."

Kevin Kelly has entered the chat. When I hear people talking about computer chips as evolving organisms, I think about What Technology Wants, the 2010 book by Kevin Kelly, where he says that technology is a lifeform that evolves, and also one which manipulates us for its own evolution. 

But the real reason we're posting this article:

The next growth spurt in transistor miniaturization and computing capability is now overdue, they say.

They're saying that Moore's Law is about to jump again, and the DRAM chips, which aren't too much different from neuromorphic architectures, are about to help us make that jump. 
[Moore's Law]

"The end of silicon chip era is in view"

via Rockefeller University:  Moore's Law Revisited through Intel Chip Density. David Burg and Jessee H Ausubel. PLOS ONE (2021). https://doi.org/10.1371/journal.pone.0256245


Artificial brain networks simulated with new quantum materials
Sep 2021, phys.org

By combining new supercomputing materials with specialized oxides, the researchers successfully demonstrated the backbone of networks of circuits and devices that mirror the connectivity of neurons and synapses in biologically based neural networks.

"Neuromorphic computing is inspired by the emergent processes of the millions of neurons, axons and dendrites that are connected all over our body in an extremely complex nervous system." -UC president and physicist Robert Dynes

The researchers' innovation was based on joining two types of quantum substances—superconducting materials based on copper oxide and metal insulator transition materials that are based on nickel oxide. They created basic "loop devices" that could be precisely controlled at the nano-scale with helium and hydrogen, reflecting the way neurons and synapses are connected. Adding more of these devices that link and exchange information with each other, the simulations showed that eventually they would allow the creation of an array of networked devices that display emergent properties like an animal's brain.

via University of California San Diego and Purdue University: Uday S. Goteti et al, Low-temperature emergent neuromorphic networks with correlated oxide devices, Proceedings of the National Academy of Sciences (2021). DOI: 10.1073/pnas.2103934118


GPUs open the potential to forecast urban weather for drones and air taxis
Oct 2022, phys.org

This paper is interesting; it talks about microscale airflow patterns in cities with high buildings. But I thought it would be a good idea to just repaste this explanation right here, for people who haven't noticed --

CPUs excel at performing multiple tasks, including control, logic, and device-management operations, but their ability to perform fast arithmetic calculations is limited. GPUs are the opposite. Originally designed to render 3D video games, GPUs are capable of fewer tasks than CPUs, but they are specially designed to perform mathematical calculations very rapidly.

And just wait, because TPUs are right on their heels.

via National Center for Atmospheric Research: Domingo Muñoz‐Esparza et al, Efficient Graphics Processing Unit Modeling of Street‐Scale Weather Effects in Support of Aerial Operations in the Urban Environment, AGU Advances (2021). DOI: 10.1029/2021AV000432


Intel launches its next-generation neuromorphic processor—so, what’s that again?
Oct 2021, Ars Technica

Unlike a normal processor, there's no external RAM. Instead, each neuron has a small cache of memory dedicated to its use. This includes the weights it assigns to the inputs from different neurons, a cache of recent activity, and a list of all the other neurons that spikes are sent to.

Also, re Intel's new chip"

Other changes are very specific to spiking neural networks. The original processor's spikes, as mentioned above, only carried a single bit of information. In Loihi 2, a spike is an integer, allowing it to carry far more information and to influence how the recipient neuron sends spikes. (This is a case where Loihi 2 might be somewhat less like the neurons it's mimicking in order to perform calculations better.)

via Intel's "Loihi 2: A New Generation of Neuromorphic Computing", 2021

Post Script:
Hiddenite: A new AI processor for reduced computational power consumption based on a cutting-edge neural network theory
Feb 2022, phys.org

Hiddenite: hidden neural network inference tensor engine.

It's an "accelerator chip" that "does neural networks better" by being better at pruned neural networks, which are also called "hidden neural networks", and by reducing memory needs, which is a big deal in the age of big data. 

And how often do we get to see RNGs in practical application in the news -- "The Hiddenite architecture (Fig. 2) offers three-fold benefits to reduce external memory access and achieve high energy efficiency. The first is that it offers the on-chip weight generation for re-generating weights by using a random number generator. This eliminates the need to access the external memory and store the weights."

They're also four-dimensional (4D) parallel processors. 

I don't see the words neuromorphic or TPU in here, but I imagine it's not too distantly related.

via Tokyo Institute of Technology: Hiddenite: 4K-PE Hidden Network Inference 4D-Tensor Engine Exploiting On-Chip Model Construction Achieving 34.8-to-16.0TOPS/W for CIFAR-100 and ImageNet, 15.4, ML Processors LIVE Q&A with demonstration, February 23 9:00AM PST, International Solid-State Circuits Conference 2022 (ISSCC 2022).

Posted by at No comments:
Labels: , , , , , , ,

Thursday, March 17, 2022

Guilty as Charged


Consciousness is clickbait, and quantum consciousness is mega-clickbait, so I usually avoid it, but fractal quantum consciousness? They got me. 

Image credit: Fractal Forums, 2018

First let's talk about fractals in real life.

I have the same thing happen in my dreams that you do. I'm trying to leave the house, and I forget something, and now I'm on a side mission to get that thing, but then I forget something else I need for the side thing, and now I'm on a side-side mission, but then I forget something else, ... and it repeats until I wake up. (I never make it out of the house.) 

This happens in real life, at least in New Jersey it does -- you're on a main road and need to make a left, but you have to first make a right, in order to get on an overpass (like a jughandle, aka Jersey left). But when you make the first right, you realize you can't turn from there to the overpass; you instead have to make another right, onto a road parallel to the first main road you were on, but now going in the opposite direction. But even still, you find you can't make a left off this road, so you have to make another right, ad infinitum.

The trajectory of your quest has collapsed into a fractal dimension, from which you might never make it back. 


This year, the recursive nature of consciousness is showing up in some interesting studies. It's beginning to look like those psychedelic images of the Mandelbrot set aren't just a good visual metaphor, they might underlie actual brain patterns, and get us closer to understanding what consciousness is. 


Fractal brain networks support complex thought
Oct 2021, phys.org

A Dartmouth study has found a new way to look at brain networks using the mathematical notion of fractals, to convey communication patterns between different brain regions as people listened to a short story.

Researchers show that brain networks organize in a similar way: patterns of brain interactions are mirrored simultaneously at different scales.

When people engage in complex thoughts, their networks seem to spontaneously organize into fractal-like patterns. When those thoughts are disrupted, the fractal patterns become scrambled and lose their integrity.

The study shows that when people listened to an audio recording of a 10-minute story, their brain networks spontaneously organized into fourth-order [fractal] network patterns. ... However, this organization was disrupted when the story's paragraphs were randomly shuffled.

"The more finely the story was shuffled, the more the fractal structures of the network patterns were disrupted,"
-Lucy Owen, first author and graduate student in psychological and brain sciences at Dartmouth, link
""
And it works just like you think it would:

The results show that the smallest scale (first-order) interactions occurred in brain regions that process raw sounds. Second-order interactions linked these raw sounds with speech processing regions, and third-order interactions linked sound and speech areas with a network of visual processing regions. The largest-scale (fourth-order) interactions linked these auditory and visual sensory networks with brain structures that support high-level thinking. 

via Dartmouth College: High-level cognition during story listening is reflected in high-order dynamic correlations in neural activity patterns, Nature Communications (2021). DOI: 10.1038/s41467-021-25876-x

3D Fractal w Fragmentarium - Adam Majewski on Fractal Forums - 2018


Can consciousness be explained by quantum physics? Research is closer to finding out
Jul 2021, phys.org

The Penrose-Hameroff theory of quantum consciousness argues that microtubules are structured in a fractal pattern which would enable quantum processes to occur.

First they created a quantum fractal by arranging electrons in a  Sierpiński triangle. But now they're using photonics to watch the electrons move in real time. And this means that quantum fractals behave differently than classical fractals. So now they think it's time to revisit.

via Cristiane de Morais Smith and Xian-Min Jin at Shanghai Jiaotong University: Xu, XY., Wang, XW., Chen, DY. et al. Quantum transport in fractal networks. Nat. Photon. (2021).



Now that you've been primed on the potential fractal nature of consciousness, it's time to enter the n-dimensional world. 

This next study should make your head spin, literally --


New research finds that collective neural activity is shaped like the surface of a doughnut
Jan 2022, phys.org

We already know about grid cells, they were discovered not long ago. Grid cells are the types of brain cells that map where you are in space -- your brain keeps a map in your head that's compressed by a layer of hexagonal grid coordinates. 

But now, they found that the grid itself is not a never-ending expanse of hexagons that surrounds us in two dimensions. Instead, it's a grid superimposed on a toroid (but you might call it a donut). That means the map is not two-dimensional, but multidimensional. 

This is because the grid cells do not form as a result of our motor activity as we travel over the  two-dimensional surface of the Earth. Instead these cells form based on their own innate tendencies to arrange in a way that represents a toroid more than a flat grid. Note this is the shape of the data we're talking about, not the shape of the cluster of cells themselves. It's the way the cells interact, not how they're actually laid out. (Kind of like thinking of the difference between actual distance and Hamming distance.)

The big deal though, is that it hints to us how the brain orchestrates all these subregions, coordinating together to create the complexities of higher-order functioning (the kind referenced above as "4th order"). It lies in the network structures, and in this case, those structures are part of continuous attractor networks. (They did get a lot of help from a new tool called Neuropixels, which allows access to raw output from neurons from all over the brain, all at the same time.)

So network theory will become a bigger part of understanding how the brain works. And meanwhile, we can just trip out on the idea that even when walking in a straight line, our brain is superimposing that data on a toroid model. 

Tl;dr -- The brain thinks the landscape is a toroid. (Even in your dreams; or especially in your dreams).

via Norwegian University of Science and Technology's Kavli Institute for Systems Neuroscience: Richard J. Gardner et al, Toroidal topology of population activity in grid cells, Nature (2022). DOI: 10.1038/s41586-021-04268-7


Further Reading:

Isaac Asimov's Robot Dreams -- a robot named Elvex (LVX-1) is updated with "fractal geometry" because the offending young scientist though it would "produce a brain pattern with more complexity, possibly closer to that of a human". The robot begins to dream about self-preservation, in direct opposition to the Laws of Robots, and is subsequently killed ("killed"?).

Maertens, James W. , Donald E. Palumbo. "Chaos Theory, Asimov's Foundations and Robots, and Herbert's Dune: the Fractal Aesthetic of Epic Science Fiction." Utopian Studies, vol. 14, no. 1, winter 2003, pp. 244+. Penn State University Press. https://www.jstor.org/stable/20718595

The Hyperbolic Geometry of DMT Experiences at the Harvard Science of Psychedelics Club in the year 2020, with Andrés Gómez Emilsson from the Qualia Research Institute

Quantum Fractals, 2019

Posted by at No comments:
Labels: , , , , , , , ,

Tuesday, July 13, 2021

Artificial Intelligence is Human After All

Medical AI models rely on 'shortcuts' that could lead to misdiagnosis of COVID-19
Jun 2021, phys.org

Very interesting lesson for us humans, in terms of profiling and stereotypes:
The team found that, rather than learning genuine medical pathology, these models rely instead on shortcut learning to draw spurious associations between medically irrelevant factors and disease status. Here, the models ignored clinically significant indicators and relied instead on characteristics such as text markers or patient positioning that were specific to each dataset to predict whether someone had COVID-19.

via  University of Washington: AI for radiographic COVID-19 detection selects shortcuts over signal, Nature Machine Intelligence (2021). DOI: 10.1038/s42256-021-00338-7 

Computer scientist researches interpretable machine learning, develops AI to explain its discoveries
Nov 2020, phys.org

Finally a deep learning machine that can explain how it got its results (something previously not available, hence the term black box AI). Or is this just mansplaining? Nipsplaining.

This Looks Like That: Deep Learning for Interpretable Image Recognition, Chaofan Chen et al, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.

Figure 1: Image of a clay colored sparrow and how parts of it look like some learned prototypical parts of a clay colored sparrow used to classify the bird’s species. -source

DeepMind's AlphaZero breathes new life into the old art of chess
Sep 2020, phys.org

Maybe disturbing? We need the robot to help us learn to be human again? To be ... something? Again?

On the topic of playing chess vs practicing chess, and of course, on being human (vs being a computer) -- As chess grandmaster Vladimir Kramnik recently told Wired magazine, "For quite a number of games on the highest level, half of the game—sometimes a full game—is played out of memory. You don't even play your own preparation; you play your computer's preparation."

The solution? Change the game. With the help of AlphaZero, they discovered new variations on the game of chess that would force players to play again for the first time, I guess. Such changes made were to forbid castling, or introduce self-capture, or allow pawns to move two spaces at once. We evolve together. 

Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess, arXiv:2009.04374 [cs.AI] 

Post Script:
Research finds some AI advances are over-hyped
June 2020, phys.org
An article in Science magazine assessing the study cites a meta-analysis of information retrieval algorithms used in search engines over a decade though 2019 and found "the high mark was actually set in 2009." Another study of neural network recommendation systems used by streaming services determined that six of the seven procedures used failed to improve upon the simpler algorithms devised years earlier.
But isn't the new wave of neural networks about size? The algorithms themselves are rather simple, but it's the size of the network of gpu's times the size of the dataset that make the system so powerful.

Also, this sure sounds like an argument against the big dogs in tech inhibiting innovation through monopolistic control. 
Posted by at No comments:
Labels: , , , ,

Saturday, July 11, 2020

Quantum Simulacrum


Teaching physics to neural networks removes 'chaos blindness'
July 2020, phys.org

Now the robots go to school to get an education, in this case, physics.

Must drop Vernor Vinge's 2006 Rainbows End, which describes the physics program that inform the simulated worlds of the future, making them very convincing. Some simulations have crappy physics packages, and glitch-out, disobeying classical mechanics etc.

-image source: Cosmic Quasar - Daria Sokol MIPT Press Office
Posted by at No comments:
Labels: , , , , ,

Monday, February 17, 2020

Deep Creep

Visualizing and Understanding Convolutional Networks -- This goes back to 2013 already, but I recently came across these images, and they are so mesmerizing I had to archive them.

The images pasted here are from a paper about convolutional neural networks. The researchers are able to train a network with tons of images, and then ask that network to classify new images it hasn't seen yet. Getting the detection error rate down to zero is the goal. This one does a good job.

But what makes this report special, is that we get to see how the system spits back what the different "neurons" see. The network develops layers or clusters that recognize different things; some are good at low-level features like lines and edges, and some are good at high level things like "bicycles" or "origami." Together they learn how to see.


^This is the first layer, it sees angles and colors.


^This is the second. This one's getting more complicated patterns. Notice the similarities, but also the differences. Of the 3x3 sets, which one is not like the other? The network saw all of those as similar. The network says that those images sit close together in image-space (the total space of all possible images, or at least all the images it was trained on).

It's a game to try and figure out the common denominator. We don't really know the common denominator, and we can't know, because we're just not computers. This is why they call it "deep." In the middle of this network, deep in there, is a layer with information that is just too complex for us. Collapse 33,000 dimensions into 3, and now try and communicate features of the 33,000 using only those 3 points of information. Can't do it. That's why it's mysterious.


^Now the third layer. Ok, so I see the people, I see the barcode/text motif, the honeycomb/diamond clusters, even the lady bug and the tomato -- it's a stretch, but I get it. But some of these are just nuts. White lower right corners? It sees white lower right corners?


^Fourth layer. No idea what this thing is talking about.


^Fifth layer. Here we go, people, dogs, flowers, now it's all making sense. And that was the point of creating this network -- you give it any picture of a flower, no matter how weird of a picture, barely looks like a flower, and this network will recognize it and classify it as a flower. (Mostly; it's not perfect.)

But there comes a point in the middle there, we have no idea what this thing is thinking about. And we never will. Just like other people, and how we can never really know what someone else is thinking. Which is to say that the computers are now a lot more like other people than they ever used to be. They have become mysterious.

Notes:
Zeiler and Fergus, Visualizing and Understanding Convolutional Networks, 2013 [ZFNet].
https://arxiv.org/abs/1311.2901

Posted by at No comments:
Labels: , , , ,

Sunday, December 8, 2019

Light Hype, Crystal Prediction, and Cerebral Diamonoids


Energy-free superfast computing invented by scientists using light pulses
May 2019, phys.org

Researchers demonstrate all-optical neural network for deep learning
Sep 2019, phys.org

Researchers teleport information within a diamond
June 2019, phys.org

Diamonds in your devices - Powering the next generation of energy storage
Dec 2019, phys.org

Boron-doped nanodiamond to be specific.

Crystal with a twist - scientists grow spiraling new material
Jun 2019, phys.org

"No one expected 2-D materials to grow in such a way. It's like a surprise gift," said Jie Yao, an assistant professor of materials science and engineering at UC Berkeley.

"While the shape of the crystals may resemble that of DNA, whose helical structure is critical to its job of carrying genetic information, their underlying structure is actually quite different. Unlike "organic" DNA, which is primarily built of familiar atoms like carbon, oxygen and hydrogen, these "inorganic" crystals are built of more far-flung elements of the periodic table, in this case, sulfur and germanium. And while organic molecules often take all sorts of zany shapes, due to unique properties of their primary component, carbon, inorganic molecules tend more toward the straight and narrow."

Cyborg organoids offer rare view into early stages of development
Aug 2019, phys.org

"If we can develop nanoelectronics that are so flexible, stretchable, and soft that they can grow together with developing tissue through their natural development process, the embedded sensors can measure the entire activity of this developmental process," said Jia Liu, Assistant Professor of Bioengineering at SEAS and senior author of the study. "

Brain waves detected in mini-brains grown in a dish
Sep 2019, phys.org

World first as artificial neurons developed to cure chronic diseases
Dec 2019, phys.org

 Optimal solid state neurons, Nature Communications (2019).
DOI: 10.1038/s41467-019-13177-3 


***
We will all live inside diamonds.
Optical intelligentities in neuromorphic cerebral organoid diamonds, to be specific.

***

Scientists create a 'crystal within a crystal' for new electronic devices
Dec 2019, phys.org

Storing data in everyday objects
Dec 2019, phys.org
A method for marking products with a DNA "barcode" embedded in miniscule glass beads -- These nanobeads are used in industry as tracers for geological tests or as markers for high-quality food products, thus distinguishing them from counterfeits using a relatively short barcode consisting of a 100-bit code. This technology has now been commercialized by ETH spin-off Haelixa. 
They call the storage-form "DNA of Things" 
"All other known forms of storage have a fixed geometry: A hard drive has to look like a hard drive, a CD like a CD. You can't change the form without losing information," Erlich says. "DNA is currently the only data storage medium that can also exist as a liquid, which allows us to insert it into objects of any shape." 
A further application of the technology would be to conceal information in everyday objects, a technique experts refer to as steganography. 
Grass, Erlich and their colleagues used the technology to store a short film about this archive (1.4 megabytes) in glass beads, which they then poured into the lenses of ordinary glasses. "It would be no problem to take a pair of glasses like this through airport security and thus transport information from one place to another undetected," Erlich says. In theory, it should be possible to hide the glass beads in any plastic objects that do not reach too high a temperature during the manufacturing process.
Substance found in fossil fuels can transform into pure diamond
Mar 2020, phy.org

Posted by at No comments:
Labels: , , , , , , , , , , ,
Subscribe to: Posts (Atom)