Commit Graph

60 Commits

Author SHA1 Message Date
81d7c7027c [*] Cont work on AuIOHandle cleanup 2024-03-16 09:51:41 +00:00
9a846a6d2f [*] Update IAsyncFileStream::NewTransaction 2024-03-09 01:12:57 +00:00
a9fd9221b7 [+] AuFS::OpenAsyncFromSharedHandle 2024-03-09 00:56:59 +00:00
1e10d95b0b [*] Update TryAttachToCompletionGroup under NT file stream 2024-01-06 22:23:56 +00:00
3766ea8b86 [+] IAsyncTransaction::GetCompletionGroup
[*] IO improvements
2024-01-06 04:32:54 +00:00
d199b52c0f [*] Improved support for completion groups when used with net streams 2024-01-02 07:06:42 +00:00
662dbac0c1 [+] IO completion groups 2023-12-28 16:49:11 +00:00
0faf4c7846 [+] Aurora::Processes::OutputOf
[+] Aurora::Processes::OutputOfAsync
[*] Fix win32 regression
2023-12-23 07:05:44 +00:00
4ce49941ff [+] SysCheckNotNullMemory
[+] SysCheckNotNull
[+] SysCheckRetExpNotNull
[+] SysCheckRetExpNotNullMemory
2023-10-24 17:43:00 +01:00
cf363b9f5b [*] Ensure this never throws 2023-08-22 09:45:32 +01:00
424cccd00a [+] IAsyncTransaction::HasCompleted 2023-08-09 00:14:36 +01:00
df6f29bf38 [+] IAsyncTransaction::SetBaseOffset 2023-08-09 00:02:35 +01:00
3ff5d8a74e [*] NT FSStream: Added more push debug errors 2023-07-29 10:13:25 +01:00
7e2aa2de3d [+] AuIO::IIOHandle 2023-07-29 08:02:47 +01:00
0cf38d5fcd [*] Fix: IPC test fails on Windows 7 2023-07-28 17:54:30 +01:00
d45dc977d8 [*] NT: Further reduce Win32 link-time requirements cont (1948dd0c) 2023-07-24 12:48:42 +01:00
4d1201a4c1 [+] EStreamForward::eNewConsoleWindow
[*] NT: Improve the selection of flags used when spawning processes
[*] NT: Caught regression in NTs FS/generic transaction IsSignaled
2023-07-06 13:44:32 +01:00
1fa433540b [*] Experimental lower-latency IO pipes
[*] revert win32 net tx hardening - do not use the Reset function to null pSocket
[*] fix a bytebuffer circular buffer path
[*] update how TLS protocol stacks can snap the stack
2023-05-11 16:05:00 +01:00
30b0c2e0ba [*] Preemptive GetOverlappedResult fast-paths under WaitMultiple2 that rely on the alertable yield under next read/write cannot be trusted if the read count is zero.
Direct IO-specific quirk?
Windows 7 specific quirk?
Direct IO under Win7 specific quirk?
idk, but this fixes it
2023-04-26 07:25:45 +01:00
c79a20c7fe [*] i hate object cache 2022-12-14 07:12:43 +00:00
bb9c383aee [+] ICompressionInterceptor::LimitPassthroughOnOverflow
[+] IOPipeRequest::uMinBytesToRead
[+] (secret api intended for unix users) AuIO::NewLSOSHandleEx
[*] Fix quirks when running Gtk under an io processor
...CtxYield shouldn't spin while work (improper breakout on remote update)
[*] EFileOpenMode::eWrite should assume O_CREAT semantics making eReadWrite somewhat redundant. OpenWrite + eWrite should reasonably work with file-appends. it should not mean force create + cucked GetLength().
2022-11-20 10:31:13 +00:00
67905a4192 [+] Network + Protocol + TLS - Initial Commit
=============================================================================
Network ]====================================================================
=============================================================================

[+] Added (very) early Aurora::IO::Net implementation
[+] AuNet::EHostnameType
[+] AuNet::EIPProtocol
[+] AuNet::ENetworkError
[+] AuNet::ETransportProtocol
[+] AuNet::INetInterface
[+] AuNet::INetSrvDatagram
[+] AuNet::INetSrvResolve
[+] AuNet::INetSrvSockets
[+] AuNet::INetSrvWorkers
[+] AuNet::INetWorker
[+] AuNet::IPAddress
[+] AuNet::IResolver
[+] AuNet::ISocket
[+] AuNet::IResolver
[+] AuNet::ISocketBase
[+] AuNet::ISocketChannel
[+] AuNet::ISocketDriver
[+] AuNet::ISocketDriverFactory
[+] AuNet::ISocketServer
[+] AuNet::ISocketServerDriver
[+] AuNet::NetEndpoint
[+] AuNet::NetError
[+] AuNet::NetHostname
(+implementation)

=============================================================================
Protocol ]===================================================================
=============================================================================

[+] IProtocolInterceptor
[+] IProtocolInterceptorEx
[+] IProtocolStack
(+implementation)

=============================================================================
TLS ]========================================================================
=============================================================================

[+] ITLSContext
[+] TLSProtocolRecv
[+] TLSProtocolSend
(+implementation)

=============================================================================
IO Bug Fixes ]===============================================================
=============================================================================

[*] IOProcessor::SubmitIOWorkItem should signal the CvEvent, forcing at least once future tick (wont optimize with if in tick & not yet dispatched work items)
[*] Split IOPipeWork in into IOPipeProcessor header
[+] IOPipeWork::GetBuffer (internal reallocation)
[*] Harden against IAsyncTransactions without a loop source
[*] Missing null `if (processor->listener)` in IOProcessor
[*] Solved some soft-lock conditions under Linux's LoopQueue (added deferred commits)
[*] Quick hack: IOProcessor::HasItems() should OR the early can-tick check function.

=============================================================================
Other ]======================================================================
=============================================================================

[+] Linux: LSSignalCatcher
[+] `static void AuResetMember(Aurora::Memory::ByteBuffer &ref)` for AuROXTL
[*] Attempt to enforce a normalization and don't overwrite-readptr-under-istreamwriters policy in ByteBuffer_ReadWrite (circular buffers)
[*] Bad ECC ctors

=============================================================================
Known issues ]===============================================================
=============================================================================

> Linux net is nowhere near done
> UDP socket emulation layer isn't implemented
> Ciphersuite API is a stub
> Private key API is a stub
> ...therefore no TLS servers
> Missing thread safety precautions under net
> Net implementation is still beri early
2022-08-28 20:02:06 +01:00
42b364840e [*] Cleanup Async.NT.cpp
[*] Fix bug in IOProcessor where work items were not enough to cont execution in a loop checking ::HasItems
[+] PromiseCallbackFunctional constructor with one functor arg
2022-08-18 17:19:32 +01:00
13480a0473 [*] NT bug fix: pipe breakages were occurring as a non-block instant readfileex operation, and didn't simulate an async read 0 success.
[*] IOProcessor's stop method now returns true on success instea of true on delegated stop
2022-08-08 16:32:45 +01:00
8fb7b7e1ee [*] IOPipeProcessor improvements
[*] NT async file: error 38 "Reached the end of the file." should be handled as a zero length packet much like ERROR_BROKEN_PIPE for stream read interface consistency
2022-07-20 15:10:07 +01:00
7fb73ccdb4 [+] IPC Logger: Added optional stream frame header (single u32) option
[*] FIX: Missing `this->isStream = isStream;`
[*] FIX: Blocking APIs under async objects under NT were using illegal calls that were for some reason not failing
2022-07-05 20:48:33 +01:00
97069a1515 [*] Undo fail on pipe broken, restore bytes written = zero behavior for reusable pipes. Will match on Linux. 2022-07-05 04:17:28 +01:00
c0ee48435c [*] More IO bug fixes; a double free, statement that should be uncommented, and wrong IO handle under Async.NT.cpp read 2022-07-04 23:42:02 +01:00
f8c45613a1 [*] I'm a dumbass. This was accounted for a long ass time ago 2022-06-29 21:50:37 +01:00
bc1296a35c [*] Deprecate NT impl of latching completeness of async transactions 2022-06-29 19:47:32 +01:00
250d3fd11e [*] Passive crinkling of today : Async.NT.cpp can now be aborted. Will port identical behavior to Linux. 2022-06-24 20:33:55 +01:00
3a76aa6d69 [*] Fix IOPipeProcessor was checking against base + length instead of write head to determine stream consumer completion
[*] Fixup IPCPipe.NT recycling, including proper disconnect acknowledgment and removal of atomic release hack
[*] End of stream callback hack in Async.NT for IPC. net requires something else; this hack will do
[*] Fix io spin: bShouldReadNext wasn't reset on end of frame
2022-06-22 21:24:04 +01:00
7fde7d04fb [+] EPipeCallbackType
[+] IIOBufferedStreamAvailable callback
[+] IIOProcessor singleshot work items / IIOProcessorWorkUnit
[+] IOPipeCallback description of a pipes destination
[+] IOPipeInputData description of a pipes source
[+] IOPipeRequest, IOPipeRequestAIO, IOPipeRequestBasic
[+] IPipeBackend hooks for on start/end hooks of IOPipeRequestBasics
[*] Update IOAdapaterAsyncStream implementation to better support caller buffering
[*] Updated IAsyncStreamReader to include a warm/dequeue API for direct async usage
[*] Fix NT IO regressions
[*] Fix ThreadPool shutdown on an unregistered thread
[*] Fix race condition in Async.NT.cpp & fix signalable state to closely match Linux (dunno how this was passing before)
[*] Refactor IOProcessorWorkUnit -> IIOProcessorWorkUnit
[*] Update experimental header to include the changes
2022-06-21 05:49:36 +01:00
5d2b642ac0 [*] Huge refactor - Move Aurora::[IPC and Loop] to Aurora::IO 2022-06-12 01:29:28 +01:00
44108a322e 2/3 of the IO update (very early implementation)
[+] TTYConsole::GetPaddingTopOfLog,GetPaddingHeadOfLog,GetPaddingTopOfLog [+ set variants]
[+] IO::IOYield()
[+] IO::IAsyncTransaction::Failed,GetOSErrorCode()
[+] IByteBufferStreamPair
[+] IIOBufferedInterceptor
[+] IIOBufferedProcessor
[+] IIOEventListener
[+] IIOPipeEventListener
[+] IIOProcessorEventListener
[+] IIOProcessorManualInvoker
[+] IIOWaitableIOLoopSource
[+] IIOWaitableIOTimer
[+] IIOWaitableItem
[+] IIOWaitableTickLimiter
[+] IOAdapterAsyncStream
[+] IOAdapterByteBuffer
[+] IOAdapterCompression
[+] IOAdapterSeeking
[*] Cleanup CpuInfo.Linux.cpp
[*] Fixup async threadpool some more
[*] LSTimer.NT.cpp updates timer object on tick state update, akin to Linux
2022-06-12 00:01:27 +01:00
e29843dc7e [*] Beyond dumb fio offset issue 2022-06-06 22:34:49 +01:00
86ac108f11 [*] Move AuIO::WaitMultiple
[+] AuIO::WaitMultiple2
2022-05-13 19:25:29 +01:00
61f20b6c69 [+] Linux: missing new IAsyncTransaction::Reset method 2022-05-05 12:48:37 +01:00
dfc21453ef [+] Add IAsyncTransaction::Reset
[*] Bug fixes: Async.NT.cpp
2022-05-05 11:24:44 +01:00
97cdbc9f27 [+] Overlapped stdout/in/err processes API 2022-05-03 11:10:04 +01:00
371006c04e [+/*] NT IPCPipe, updated interface for bidirectional comms. Introducing linux regression 2022-04-17 13:40:08 +01:00
7a0d9701b5 [*] NT regression and fixes 2022-04-15 19:20:12 +01:00
415116c891 [+] Linux Async IO 2022-04-13 12:00:35 +01:00
4617587adb [*] Async file aio API WaitMultiple should return a DWORD and call the callbacks 2022-04-12 20:17:58 +01:00
cf70f0d45c [*/+/-] MEGA COMMIT. ~2 weeks compressed.
The intention is to quickly improve and add util apis, enhance functionality given current demands, go back to the build pipeline, finish that, publish runtime tests, and then use what we have to go back to to linux support with a more stable api.

[+] AuMakeSharedArray
[+] Technet ArgvQuote
[+] Grug subsystem (UNIX signal thread async safe ipc + telemetry flusher + log flusher.)
[+] auEndianness -> Endian swap utils
[+] AuGet<N>(...)
[*] AUE_DEFINE conversion for
        ECompresionType, EAnsiColor, EHashType, EStreamError, EHexDump
[+] ConsoleMessage ByteBuffer serialization
[+] CmdLine subsystem for parsing command line arguments and simple switch/flag checks
[*] Split logger from console subsystem
[+] StartupParameters -> A part of a clean up effort under Process
[*] Refactor SysErrors header + get caller hack
[+] Atomic APIs
[+] popcnt
[+] Ring Buffer sink
[+] Added more standard errors
        Catch,
        Submission,
        LockError,
        NoAccess,
        ResourceMissing,
        ResourceLocked,
        MalformedData,
        InSandboxContext,
        ParseError

[+] Added ErrorCategorySet, ErrorCategoryClear, GetStackTrace
[+] IExitSubscriber, ETriggerLevel
[*] Write bias the high performance RWLockImpl read-lock operation operation
[+] ExitHandlerAdd/ExitHandlerRemove (exit subsystem)
[*] Updated API style
        Digests
[+] CpuId::CpuBitCount
[+] GetUserProgramsFolder
[+] GetPackagePath
[*] Split IStreamReader with an inl file
[*] BlobWriter/BlobReader/BlobArbitraryReader can now take shared pointers to bytebuffers. default constructor allocates a new scalable bytebuffer
[+] ICharacterProvider
[+] ICharacterProviderEx
[+] IBufferedCharacterConsumer
[+] ProviderFromSharedString
[+] ProviderFromString
[+] BufferConsumerFromProvider
[*] Parse Subsystem uses character io bufferer
[*] Rewritten NT's high perf semaphore to use userland SRW/ConVars [like mutex, based on generic semaphore]
[+] ByteBuffer::ResetReadPointer
[*] Bug fix bytebuffer base not reset on free and some scaling issues
[+] ProcessMap -> Added kSectionNameStack, kSectionNameFile, kSectionNameHeap for Section
[*] ProcessMap -> Refactor Segment to Section. I was stupid for keeping a type conflict hack API facing
[+] Added 64 *byte* fast RNG seeds
[+] File Advisorys/File Lock Awareness
[+] Added extended IAuroraThread from OS identifier caches for debug purposes
[*] Tweaked how memory is reported on Windows. Better consistency of what values mean across functions.
[*] Broke AuroraUtils/Typedefs out into a separate library
[*] Update build script
[+] Put some more effort into adding detail to the readme before rewriting it, plus, added some media
[*] Improved public API documentation
[*] Bug fix `SetConsoleCtrlHandler`
[+] Locale TimeDateToFileNameISO8601
[+] Console config stdOutShortTime
[*] Begin using internal UTF8/16 decoders when platform support isnt available (instead of stl)
[*] Bug fixes in decoders
[*] Major bug fix, AuMax
[+] RateLimiter
[+] Binary file sink
[+] Log directory sink
[*] Data header usability (more operators)
[+] AuRemoveRange
[+] AuRemove
[+] AuTryRemove
[+] AuTryRemoveRange
[+] auCastUtils
[+] Finish NewLSWin32Source
[+] AuTryFindByTupleN, AuTryRemoveByTupleN
[+] Separated AuRead/Write types, now in auTypeUtils
[+] Added GetPosition/SetPosition to FileWriter
[*] Fix stupid AuMin in place of AuMax in SpawnThread.Unix.Cpp
[*] Refactored Arbitrary readers to SeekingReaders (as in, they could be atomic and/or parallelized, and accept an arbitrary position as a work parameter -> not Seekable, as in, you can simply set the position)
[*] Hack back in the sched deinit
[+] File AIO loop source interop
[+] Begin to prototype a LoopQueue object I had in mind for NT, untested btw
[+] Stub code for networking
[+] Compression BaseStream/IngestableStreamBase
[*] Major: read/write locks now support write-entrant read routines.
[*] Compression subsystem now uses the MemoryView concept
[*] Rewrite the base stream compressions, made them less broken
[*] Update hashing api
[*] WriterTryGoForward and ReaderTryGoForward now revert to the previous relative index instead of panicing
[+] Added new AuByteBuffer apis
    Trim, Pad, WriteFrom, WriteString, [TODO: ReadString]
[+] Added ByteBufferPushReadState
[+] Added ByteBufferPushWriteState
[*] Move from USC-16 to full UTF-16. Win32 can handle full UTF-16.
[*] ELogLevel is now an Aurora enum
[+] Raised arbitrary limit in header to 255, the max filter buffer
[+] Explicit GZip support
[+] Explicit Zip support
[+] Added [some] compressors

et al
2022-02-17 00:11:40 +00:00
7eb6900e9f [+] Added shared pointer extension, begin experimenting defining throw-on-null mechanic
[+] AuRemoveConst
[*] Support circular reference in Aurora pipelines Include, added support for early Aurora::Build AuroraEnum
[+] Added SWInfo API
[+] AU_COPY_MOVE, AU_MOVE, AU_COPY to go with AU_NO_... variants
[+] Adding GetProcessId
2022-01-26 00:22:02 +00:00
e5e36bd887 Large Commit
[*] Fix deadlock in the async subsystem (NoLockShutdown vs Shutdown in exception handler)
[+] Added ProccessMap NT variant
[+] Added ToolHelp image profiling
[*] Improved exception awareness
[*] Delegated SpawnThread to isolated TU, ready for reuse for RunAs and XNU Open - now with horrible evil alloc that could fail
[+] Added header for future api 'UtilRun'
[*] Improve NT core detection
[*] Changed small affinity bitmap to AuUInt64 instead of AuUInt32
[+] Added data structure to hold cpuids/affinity masks
[+] Implemented logger sinks
[+] Implemented logger glue logic
[*] Began migrating older loggers to sink-based default devices
[*] Minor refactors
[*] Improved internal exception discarding, not yet nothrow capable
[*] Minor create directory fix
2022-01-24 18:43:53 +00:00
c965d8384f [+] AuStaticPointerCast
[*] More refactoring
2022-01-19 18:53:22 +00:00
7759d38ed0 [*] Exterminate more std:: references 2022-01-19 17:08:13 +00:00
235225ec9d [+] Added WriteEoS
[*] Fix two build issues (missing = 0 in vtbl and messed up macros)
2022-01-17 15:21:36 +00:00