Неустранимое исключение: java.lang.OutOfMemoryError с использованием okhttp3 okio для загрузки файлов
У меня есть приложение, которое скачивает контент из Интернета. музыка, видео, PDF.... как менеджер загрузки.
Но теперь происходит сбой при загрузке контента:
E/LVN/advanced_memory_manager.c: ---------------------------------- AMM report ------------------------------
-> Memory Currently Allocated: 0 bytes <=> 0 components
-> Max Memory Need: 512000 bytes
-> Overall Memory Allocation: 515652 bytes (l:423)
E/art: Throwing OutOfMemoryError "Failed to allocate a 2060 byte allocation with 16777232 free bytes and 308MB until OOM; failed due to fragmentation (required continguous free 4096 bytes where largest contiguous free 0 bytes)"
E/AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher
Process: com.rokki.life2, PID: 32171
java.lang.OutOfMemoryError: Failed to allocate a 2060 byte allocation with 16777232 free bytes and 308MB until OOM; failed due to fragmentation (required continguous free 4096 bytes where largest contiguous free 0 bytes)
at okio.Segment.<init>(Segment.java:58)
at okio.SegmentPool.take(SegmentPool.java:46)
at okio.Buffer.writableSegment(Buffer.java:1114)
at okio.Okio$2.read(Okio.java:137)
at okio.AsyncTimeout$2.read(AsyncTimeout.java:211)
at okio.RealBufferedSource.read(RealBufferedSource.java:50)
at okhttp3.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:381)
at okio.RealBufferedSource.request(RealBufferedSource.java:71)
at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:225)
at okhttp3.RealCall$ApplicationInterceptorChain.proceed(RealCall.java:187)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:160)
at okhttp3.RealCall.access$100(RealCall.java:30)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:127)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
а также
Fatal Exception: java.lang.OutOfMemoryError: Failed to allocate a 106405148 byte allocation with 16777120 free bytes and 82MB until OOM
at java.lang.String.<init>(String.java:332)
at java.lang.String.<init>(String.java:371)
at okio.Buffer.readString(Buffer.java:579)
at okio.Buffer.readString(Buffer.java:562)
at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:244)
at okhttp3.RealCall$ApplicationInterceptorChain.proceed(RealCall.java:187)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:160)
at okhttp3.RealCall.access$100(RealCall.java:30)
at okhttp3.RealCall$AsyncCall.execute(RealCall.java:127)
at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)
Я уже добавил это в свой манифест:
<application
android:name=".app.MyApp"
android:allowBackup="true"
android:hardwareAccelerated="true"
android:icon="@mipmap/ic_launcher"
tools:replace="android:icon"
android:label="@string/app_name"
android:largeHeap="true"
android:supportsRtl="true"
android:theme="@style/AppTheme.NoActionBar">
Образец кода:
private OkHttpClient client;
onViewCreated...{
//Initiate OkHttp with interceptor
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);
client = new OkHttpClient.Builder()
.addInterceptor(logging)
.build();
...
}
private void downloadPdf() {
AsyncTask.execute(new Runnable() {
@Override
public void run() {
client
.newCall(getRequest(Config._API_PDF))
.enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
...
}
@Override
public void onResponse(Call call, Response response) throws IOException {
try {
InputStream ins = response.body().byteStream();
BufferedReader in = new BufferedReader(new InputStreamReader(ins));
String lineTotal = "";
while (true) {
String line = in.readLine();
if (line == null)
break;
else
lineTotal += line;
}
...json parsing ...
...
Мой BuildGradle:
apply plugin: 'com.android.application'
android {
compileSdkVersion 23
buildToolsVersion "23.0.2"
defaultConfig {
applicationId "com.my.app"
minSdkVersion 17
targetSdkVersion 22
versionCode 1
versionName "1.0"
ndk { abiFilters "armeabi", "x86", "mips" }
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
}
}
}
dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
final SUPPORT_LIBRARY_VERSION = '23.2.0'
final PLAY_SERVICES_VERSION = '8.3.0'
final RETROFIT_VERSION = '2.0.0'
final OKHTTP3_VERSION = '3.2.0'
compile "com.android.support:design:$SUPPORT_LIBRARY_VERSION"
compile "com.android.support:support-v4:$SUPPORT_LIBRARY_VERSION"
compile "com.android.support:appcompat-v7:$SUPPORT_LIBRARY_VERSION"
compile "com.android.support:cardview-v7:$SUPPORT_LIBRARY_VERSION"
compile "com.android.support:recyclerview-v7:$SUPPORT_LIBRARY_VERSION"
compile "com.squareup.retrofit2:retrofit:$RETROFIT_VERSION"
compile "com.squareup.retrofit2:converter-gson:$RETROFIT_VERSION"
compile "com.squareup.retrofit2:adapter-rxjava:$RETROFIT_VERSION"
compile "com.squareup.okhttp3:logging-interceptor:$OKHTTP3_VERSION"
compile 'com.google.code.gson:gson:2.6.2'
compile 'com.jakewharton:butterknife:7.0.1'
}
тогда у меня есть другой аналогичный метод для загрузки видео...
Кто-нибудь сталкивался с этим раньше? Спасибо
3 ответа
Когда вы используете HttpLoggingInterceptor.Level.BODY, то вы пытаетесь скачать большой файл, сохранит все тело в памяти для журнала.
Это легко позволить ООМ.
Попробуйте удалить тело журнала или просто зарегистрировать NONE, basic или header и попробуйте снова.
И когда вы пытаетесь получить большой файл, можете использовать @Streaming в retrofit2, как это.
@Streaming
@GET
fun downloadFileWithDynamicUrlSync(@Url fileUrl: String): Call<ResponseBody>
Попробуйте удалить тело журнала здесь.
onViewCreated...{
//Initiate OkHttp with interceptor
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);
Попробуй это.
onViewCreated...{
//Initiate OkHttp with interceptor
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.NONE);
Если спешите, скопируйте / вставьте эту версию HttpLoggingInterceptor и используйте ее как оригинал. Это решит вашу проблему, отключив слишком длинные журналы. Работает очень хорошо.
class CustomHttpLoggingInterceptor @JvmOverloads constructor(
private val maxLogSize : Long = 5000,
private val logger: Logger = Logger.DEFAULT
) : Interceptor {
@Volatile private var headersToRedact = emptySet<String>()
@set:JvmName("level")
@Volatile var level = Level.NONE
enum class Level {
/** No logs. */
NONE,
/**
* Logs request and response lines.
*
* Example:
* ```
* --> POST /greeting http/1.1 (3-byte body)
*
* <-- 200 OK (22ms, 6-byte body)
* ```
*/
BASIC,
/**
* Logs request and response lines and their respective headers.
*
* Example:
* ```
* --> POST /greeting http/1.1
* Host: example.com
* Content-Type: plain/text
* Content-Length: 3
* --> END POST
*
* <-- 200 OK (22ms)
* Content-Type: plain/text
* Content-Length: 6
* <-- END HTTP
* ```
*/
HEADERS,
/**
* Logs request and response lines and their respective headers and bodies (if present).
*
* Example:
* ```
* --> POST /greeting http/1.1
* Host: example.com
* Content-Type: plain/text
* Content-Length: 3
*
* Hi?
* --> END POST
*
* <-- 200 OK (22ms)
* Content-Type: plain/text
* Content-Length: 6
*
* Hello!
* <-- END HTTP
* ```
*/
BODY
}
interface Logger {
fun log(message: String)
companion object {
/** A [Logger] defaults output appropriate for the current platform. */
@JvmField
val DEFAULT: Logger = object : Logger {
override fun log(message: String) {
Platform.get().log(Platform.INFO, message, null)
}
}
}
}
fun redactHeader(name: String) {
val newHeadersToRedact = TreeSet(String.CASE_INSENSITIVE_ORDER)
newHeadersToRedact += headersToRedact
newHeadersToRedact += name
headersToRedact = newHeadersToRedact
}
@Deprecated(
message = "Moved to var. Replace setLevel(...) with level(...) to fix Java",
replaceWith = ReplaceWith(expression = "apply { this.level = level }"),
level = DeprecationLevel.WARNING)
fun setLevel(level: Level) = apply {
this.level = level
}
@JvmName("-deprecated_level")
@Deprecated(
message = "moved to var",
replaceWith = ReplaceWith(expression = "level"),
level = DeprecationLevel.ERROR)
fun getLevel(): Level = level
@Throws(IOException::class)
override fun intercept(chain: Interceptor.Chain): Response {
val level = this.level
val request = chain.request()
if (level == Level.NONE) {
return chain.proceed(request)
}
val logBody = level == Level.BODY
val logHeaders = logBody || level == Level.HEADERS
val requestBody = request.body
val connection = chain.connection()
var requestStartMessage =
("--> ${request.method} ${request.url}${if (connection != null) " " + connection.protocol() else ""}")
if (!logHeaders && requestBody != null) {
requestStartMessage += " (${requestBody.contentLength()}-byte body)"
}
logger.log(requestStartMessage)
if (logHeaders) {
if (requestBody != null) {
// Request body headers are only present when installed as a network interceptor. Force
// them to be included (when available) so there values are known.
requestBody.contentType()?.let {
logger.log("Content-Type: $it")
}
if (requestBody.contentLength() != -1L) {
logger.log("Content-Length: ${requestBody.contentLength()}")
}
}
val headers = request.headers
for (i in 0 until headers.size) {
val name = headers.name(i)
// Skip headers from the request body as they are explicitly logged above.
if (!"Content-Type".equals(name, ignoreCase = true) &&
!"Content-Length".equals(name, ignoreCase = true)) {
logHeader(headers, i)
}
}
if (!logBody || requestBody == null) {
logger.log("--> END ${request.method}")
} else if (bodyHasUnknownEncoding(request.headers)) {
logger.log("--> END ${request.method} (encoded body omitted)")
} else if (requestBody.isDuplex()) {
logger.log("--> END ${request.method} (duplex request body omitted)")
} else {
val buffer = Buffer()
requestBody.writeTo(buffer)
val contentType = requestBody.contentType()
val charset: Charset = contentType?.charset(StandardCharsets.UTF_8) ?: StandardCharsets.UTF_8
logger.log("")
if (buffer.isProbablyUtf8()) {
if(requestBody.contentLength()>maxLogSize)logger.log(buffer.readString(maxLogSize,charset))
else logger.log(buffer.readString(charset))
logger.log("--> END ${request.method} (${requestBody.contentLength()}-byte body)")
} else {
logger.log(
"--> END ${request.method} (binary ${requestBody.contentLength()}-byte body omitted)")
}
}
}
val startNs = System.nanoTime()
val response: Response
try {
response = chain.proceed(request)
} catch (e: Exception) {
logger.log("<-- HTTP FAILED: $e")
throw e
}
val tookMs = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNs)
val responseBody = response.body!!
val contentLength = responseBody.contentLength()
val bodySize = if (contentLength != -1L) "$contentLength-byte" else "unknown-length"
logger.log(
"<-- ${response.code}${if (response.message.isEmpty()) "" else ' ' + response.message} ${response.request.url} (${tookMs}ms${if (!logHeaders) ", $bodySize body" else ""})")
if (logHeaders) {
val headers = response.headers
for (i in 0 until headers.size) {
logHeader(headers, i)
}
if (!logBody || !response.promisesBody()) {
logger.log("<-- END HTTP")
} else if (bodyHasUnknownEncoding(response.headers)) {
logger.log("<-- END HTTP (encoded body omitted)")
} else {
val source = responseBody.source()
source.request(Long.MAX_VALUE) // Buffer the entire body.
var buffer = source.buffer
var gzippedLength: Long? = null
if ("gzip".equals(headers["Content-Encoding"], ignoreCase = true)) {
gzippedLength = buffer.size
GzipSource(buffer.clone()).use { gzippedResponseBody ->
buffer = Buffer()
buffer.writeAll(gzippedResponseBody)
}
}
val contentType = responseBody.contentType()
val charset: Charset = contentType?.charset(StandardCharsets.UTF_8) ?: StandardCharsets.UTF_8
if (!buffer.isProbablyUtf8()) {
logger.log("")
logger.log("<-- END HTTP (binary ${buffer.size}-byte body omitted)")
return response
}
if (contentLength != 0L) {
logger.log("")
logger.log(buffer.clone().readString(charset))
}
if (gzippedLength != null) {
logger.log("<-- END HTTP (${buffer.size}-byte, $gzippedLength-gzipped-byte body)")
} else {
logger.log("<-- END HTTP (${buffer.size}-byte body)")
}
}
}
return response
}
private fun logHeader(headers: Headers, i: Int) {
val value = if (headers.name(i) in headersToRedact) "██" else headers.value(i)
logger.log(headers.name(i) + ": " + value)
}
private fun bodyHasUnknownEncoding(headers: Headers): Boolean {
val contentEncoding = headers["Content-Encoding"] ?: return false
return !contentEncoding.equals("identity", ignoreCase = true) &&
!contentEncoding.equals("gzip", ignoreCase = true)
}
fun Buffer.isProbablyUtf8(): Boolean {
try {
val prefix = Buffer()
val byteCount = size.coerceAtMost(64)
copyTo(prefix, 0, byteCount)
for (i in 0 until 16) {
if (prefix.exhausted()) {
break
}
val codePoint = prefix.readUtf8CodePoint()
if (Character.isISOControl(codePoint) && !Character.isWhitespace(codePoint)) {
return false
}
}
return true
} catch (_: EOFException) {
return false // Truncated UTF-8 sequence.
}
}
}
Затем просто назовите его, как обычно:
val interceptor = CustomHttpLoggingInterceptor( )
interceptor.apply { interceptor.level = CustomHttpLoggingInterceptor.Level.BODY }
clientBuilder.addInterceptor(interceptor)
ГЛУБОКОЕ ПОГРУЖЕНИЕ:
Эта проблема хорошо известна команде OkHttp, и они не считают ее проблемой и рекомендуют разработчикам разветвить проект и реализовать свои собственные версии, подходящие для их нужд. Что имеет смысл с их стратегией проекта. подробнее здесь
Ошибка на этой линии
buffer.readString(charset) которые пытаются зарегистрировать все тело вашего запроса, ограничивая его
buffer.readString(maxLogSize,charset) мы обрезаем все строки журнала после этой ошибки OOM avoding. Просто!
Добавить нижнюю строку в defaultConfig в Gradle
multiDexEnabled true
затем в функции Android
dexOptions {
incremental = true;
preDexLibraries = false
javaMaxHeapSize "4g" // 2g should be also OK
}
Добавить эту зависимость
compile 'com.android.support:multidex:1.0.1'